How do you setup Collision for walls?

You could also just enable collisions on your wall geometry.

node.setCollideMask(BitMask32.bit(1))

But of course this will be very expensive with non-optimised geometry.
Which is why I use the octree builder from raytaller:
https://discourse.panda3d.org/viewtopic.php?t=2502&highlight=raytaller

to seriously optimise the performance of collision detection against geometry.
The only thing you need to do once you’ve got an optimised mesh is to load it from the resulting file, hide() it and enable collisions on it.

This method works beautiful for me, I have collisions against multi-k poly meshs.