I quickly hacked up some Python code that should provide a good start for anyone wanting to use gamepads, joysticks, steering wheels and the like on Linux via the joystick driver that is built into the Linux kernel (therefore negating the need for PyGame on Linux).
It directly interfaces with the Linux joystick driver via the /dev/input/js0, js1, …, jsN block devices. It opens them using a simple open() call, and uses ioctl to get metadata about the devices (such as the button and axis mapping).
Reading is implemented using a blocking loop, but this could be modified to use non-blocking I/O or you may simply decide to put the loop in a thread.
I also have a similar snippet for Windows here.
EDIT: This technique is obsolete with the new gamepad support in Panda3D 1.10.