You have to use the version of Python that Panda was compiled against. Since the version for download here has been compiled against Python 2.5 (not 2.4), it means you have to use it with Python 2.5, unless you want to get the source and build it yourself.
If you build it yourself, you can tell it to use any version of Python you’ve got.
David