I’m afraid that I can not answer this question, at least not based on any hard facts.
It seems you don’t need physics, but only collision detection. I assume you do collision response yourself. Neither Bullet nor PhysX have been designed with this in mind. PhysX probably has more function you might find useful, but current development focus is on Bullet. This is why I would go for Bullet.
Or probably stick with the native Panda3D collision system until you find something that definitely can not be handled by this system. Often it is just not being familiar enough with something that leads to the (false) impression that something is unstable.
Sorry, but it is impossible to give solid advice here.