I wonder if Collision animations would be an acceptable middle ground. Remove any collision which is not a full, face on collision, and replace it with an animation of your character sidestepping the obstruction
In other words, any time you would hit a smaller creature or player, you would sidestep them instead, but you would still collide with larger creatures and... well... walls.
I like that. And it's done before.
If I remember correctly Assassins Creed has a similar system.
If you run you'll bump into NPCs and if you walk your PC evades NPCs blocking your way.
However. For now, removing the collision detection improves the games playability.