Pointer Textures

With the introduction of Panda3D 1.7.0, there is now a very powerful but also very dangerous feature that is now available to the public.  This feature has been used internally at Walt Disney Imagineering for some time now, but it is now available to everyone.  We call these Pointer Textures.  It allows a user within python to give a long int to Panda3D, and Panda will cast this to a pointer and without question, upload the data at this pointer to the graphics card!

You can see why this is dangerous.  This bypasses any checks or copies within Panda and is a direct gate to the graphics card.  This means that if you do something wrong, your application will crash.  No asserts, no error messages, it will just flat out crash.  That should be about the extent of it, but I’d be lying if I said I didn’t cause a few BSODs here and there with this technique.   Now that we have this little disclaimer out of the way, what is it good for?

For starters, it’s really fast.  The current MovieTexture implementation actually does three copies of the same data within memory.  Once from the decoder to system memory, once to Panda safe memory, and then another to the graphics card.  This certainly works and the performance is quite reasonable.  Where this fails is when you have large amounts of video data like 1080p content. That’s a lot of data to be unnecessarily moving around.  For these applications, we use Pointer Textures.

We have an external python module that decodes movies using DirectShow and through pointer textures, we can display it in Panda as a texture.  This has side benefits as well.  Since DirectShow is multithreaded and a python module is not locked to the main python thread, DirectShow can decode the movies using different cores.  This makes it a truly multi-threaded application. On a machine with 4 cores, I’ve managed to squeeze two simultaneous 720p videos through without problems.

Since pointer textures allow you direct access to graphics card memory, it also means that you can load anything as long as you have the pointer to the data and the correct format.  Using this we’ve implemented Image Based Lighting via HDR textures.

Another added benefit to using pointer textures is that you don’t have to have to modify Panda to get special features.  For instance, if you wanted to load an image format that is not currently supported by the texture loader, you would have to setup a Panda build environment, get the third party libs, and then modify the texture loader.  Now you don’t have to.  Just compile a python module via Swig or via the C Python API and make a function that returns the pointer as a long integer.

This goes far beyond static images or movies as well.  Since it’s just data at a pointer, this method also works with special devices.  Using this, we have connected Panda to specialized cameras, webcams, TV Tuner cards and even capture cards.  The possibilities are endless.

If you want to start using this, here is a quick example that just loads a HDR pfm texture and displays it. It only works on Windows with Panda3D 1.7.0. I’ve included the source code for the python module as well if you are feeling like you want to compile that too. Simply run PfmTexture.py and you’ll should see the demo.

10 Comments »

  1. ‘This demo only for Panda 1.7.0 on 32bit windows only’…

    Comment by Anon — February 6, 2010 @ 3:46 am

  2. Long story short: this technology allows you to access low-level hardware features through Panda and use those even if Panda itself isn’t capable of that. Is that correct, in general?

    Sounds interesting, although i’m missing visible examples of application. Too bad the sample code is limited to Windows platforms.
    I’m very happy to see Panda in conjunction with such action research. It supplies Panda with a very positive image, in my eyes.

    Comment by Nemesis#13 — February 6, 2010 @ 12:15 pm

  3. can we get this multi platform and integrated into panda3d?

    Comment by treeform — February 6, 2010 @ 1:37 pm

  4. The pointer textures methods have been published and is indeed cross platform capable. It has been tested on Ubuntu 32bit as well as x64 Windows. The example is for 32bit windows because the Python Module was only compiled for that platform. Since it’s python code, it can easily be compiled in any platform that has a compiler and the python libs. I’ve included the .cpp to do so.

    Long story short: Yes, this allows you access to low-level hardware through Panda even if Panda doesn’t support it through the use of python modules. Since python modules are so easy to generate now via things like Swig or Boost.Python, you can access the power of Panda 3D and combine it with whatever hardware or software SDK’s you want without having to recompile Panda.

    This will probably be more useful to universities and experimental game developers at first. To date, I have seen this feature used with depth sensing cameras, multi-touch tables, advanced computer vision libraries, advanced augmented reality libraries.

    Comment by Bei — February 6, 2010 @ 7:02 pm

  5. […] Panda3D has a prettier website, it released version 1.7.0 and apparently has a web-plug in. The new version makes it easier to crash the computer, but also gives a performance boost – with the magic of ‘pointer textures‘. […]

    Pingback by Open Source 3D Game Engines Updates | Kerolab — February 7, 2010 @ 10:32 am

  6. […] Panda3D has a prettier website, it released version 1.7.0 and apparently has a web-plug in. The new version makes it easier to crash the computer, but also gives a performance boost – with the magic of ‘pointer textures‘. […]

    Pingback by All Zone Info – Open Source 3D Game Engines Updates « One for All Info — February 27, 2010 @ 5:50 pm

  7. Hi Bei.

    I’m a little confused about this feature. Checking on CVS, the ability to manipulate textures directly has been there since 2005, and I’ve been using it for quite some time, too (from C++). Is this a new feature because the setRamMipmapPointerFromInt one-liner wrapper allows people to use the call from Python? Has anything been done that would give me better performance than using, say, Texture::modify_ram_image? (Assuming I need no external storage.)

    So I take it that this isn’t really a performance improvement for the C++ coder, but rather it just allows a python user to reference a external storage, right? Just wanted to confirm this.

    Comment by gogg — March 3, 2010 @ 10:56 am

  8. Hi Gogg,
    What you say is true. From C++ you’ve been able to do this for some time through the use of modify_ram_image and in Python using ModifyRamImage. This method returns a pointer that you can then modify and/or copy data too. At the next draw call, Panda3D uploads this texture to the graphics card.

    Due to this fact, it is slower than using pointer textures. As an example, lets say you have an HD camera. Most of camera SDKs will return a pointer that contains the data at that frame. If you were to use ModifyRamImage, you would have to then copy the data to that ram image before Panda uploads it. If you use pointer textures, it will upload to the graphics card directly from the camera SDK. This saves you an extra copy step. Granted memcpy() in C is pretty fast, but this still happens on the CPU and is a significant hit when dealing with enough data like a full 1080p RGBA stream.

    Lastly, as you said. Yes, this also gives python users the same kind of power and performance for fast texture uploads. Hope that answers your question.

    Comment by Bei — March 4, 2010 @ 5:21 am

  9. Hi Bei,

    I`m currently using DirectShow and Panda simultaneously by rendering to a separate video window on top of the Panda window. In your post, you talked about having an own directShow module; how did you get your hands on the pointer of the decoded video to use it as a texture in Panda?
    If I managed to do that too, I could really integrate the video in my 3D world instead of just putting it on top.

    thx alot!

    Comment by smirk — July 20, 2010 @ 9:00 am

  10. I see a lot of potential for animating environments by having many lower res videos looping, or even streaming casually, as the player marches through.

    Comment by final_fantasy_fan_man — June 23, 2011 @ 2:11 am

RSS feed for comments on this post. TrackBack URL

Leave a comment