Note that in the past, it was very difficult to build Panda3D. Things have improved. It is now fairly straightforward to download and compile panda. In particular, this is a sensible thing to do if you wish to add some functionality to panda.
To avoid possible consistency problems, the documentation for building the source is included with the source, not in the manual. Download a source package from the panda website, then look at a file called README.md. It contains instructions for invoking the build system, makepanda.
A link to the latest build instructions can be found here:
While building you'll get a list of not found dependencies. Further descriptions of those can be found here: Dependencies.