The usual way to get around this is to create a bunch of nodes for each “sector”. Eg. you have a sector node at (0, 10000), to which the scene model is parented, and whenever you enter that sector, you reparent the camera to it.
This has the simple effect that Panda is only dealing with very small relative transformations when figuring out where the object is relative to the camera, even if the transformation is very large relative to render.