I tried compiling panda myself once as non-root and the building was not completed. Only when I build it as root would the build process be completed. This was on FC5.
I don’t know if it is possible to indicate that panda should be installed in a different location than /usr/…