This message is directed at Craig, who asked about this question on my recent blog post:
I’ve been so far unable to reproduce the reported problem. setOneShot() appears to be working for me in conjunction with RTMCopyRam. Here is some sample code that demonstrates it working–load it up and press “b” to create a buffer and start things going. It should render a scene offscreen and, when the result gets back to RAM, it should save it to tex.png and also display it onscreen. Does this code work for you, or does it fail?
David, I have a tangential problem. Opening an offscreen buffer using base.graphicsEngine.makeOutput is crashing under both gl and dx9 when I enable the new threading model using ‘threading-model Cull/Draw’. This is on a GTX 260/winxp 32 bitmachine.
I believe I have the same problem with the buffers.
Is this the sort of error you get?
Traceback (most recent call last):
File "C:\work\sp1\Panda3D\direct\showbase\ShowBase.py", line 1719, in __igLoop
self.graphicsEngine.renderFrame()
AssertionError: pipeline_stage == 0 at line 667 of c:\work\panda3d\panda\src\display\displayRegion.cxx
:task(error): Exception occurred in PythonTask igLoop
Traceback (most recent call last):
File "client.py", line 564, in <module>
game.run()
File "C:\work\sp1\Panda3D\direct\showbase\ShowBase.py", line 2644, in run
self.taskMgr.run()
File "C:\work\sp1\Panda3D\direct\task\Task.py", line 502, in run
self.step()
File "C:\work\sp1\Panda3D\direct\task\Task.py", line 460, in step
self.mgr.poll()
File "C:\work\sp1\Panda3D\direct\showbase\ShowBase.py", line 1719, in __igLoop
self.graphicsEngine.renderFrame()
AssertionError: pipeline_stage == 0 at line 667 of c:\work\panda3d\panda\src\display\displayRegion.cxx
Another curious side-effect is that by setting a window icon, Panda becomes unable to open a window. For example:
loadPrcFileData('', 'icon-filename icon.ico')
Specifically, Python will sit idle with no CPU usage when it hits line 611 of ShowBase.py, the graphicsEngine.makeOutput call.
OK, it’s true that it doesn’t work in multithreaded mode, and I’m working on that right now.
But I was hoping to answer whether it works in the original, single-threaded mode. Craig’s report seems to suggest that it doesn’t work even in that case. Can anyone else confirm this?
In my case it works just fine in the single-thread mode, albeit slower than threads disabled which is to be expected.
Edit: By “it” I meant my game, but I also just tried the example you posted, David, and it works as well.
Running your code from the first post on my older mac, it runs until I hit b, then if freezes for a while and crashes with a bus error:
DirectStart: Starting the game.
Known pipe types:
osxGraphicsPipe
(all display modules loaded.)
Made GLGraphicsBuffer
/var/folders/M4/M4oCK5IpHAucjm-kozc7uk+++TI/Cleanup At Startup/untitled text-339307867.848.command: line 3: 252 Bus error /usr/local/bin/python /private/var/folders/M4/M4oCK5IpHAucjm-kozc7uk+++TI/Cleanup\ At\ Startup/untitled\ text-339307867.730
logout
[Process completed]
Anyway, thats not the computer I usually use (horrible graphics support), and your code works fine on the machine which I was producing my issue on (a newer mac).
This modified version which reuses the same buffer only works the first time, then all repeated attempts never finish. This is the issue I was having I think.
Ah, hmm. This is actually beyond the original intention of setOneShot(), which was to permanently disable the buffer after the first time.
Actually, the original intention of setOneShot() was to delete the buffer after one frame, but that turned out to be problematic, so we eventually softened it into disabling the buffer after one frame, and required the user to explicitly delete it later. But since we have made that change, it makes sense to allow the buffer to be re-used again later if desired. I’ll see about making the necessary changes to support that.
FYI, I’ve committed my recent fixes for these issues, and the buildbot server has picked them up. In particular: (1) the above code, and offscreen buffers in general, should now work correctly in threaded mode, and (2) setOneShot() is now reusable.
I tried it out on the “Teapot on TV” sample and got about a +50% FPS boost, pretty good for a sample that has almost nothing going on!
My game still freezes before it gets to render a frame, I’ll try to make a small example which fails.
Ah, I see what’s going on. Not precisely a bug in Panda–the problem is that your GeomVertexWriter objects are still in scope at the time you call run(), so you are still holding the write locks on the vertices, and when the renderer tries to grab the write locks–deadlock.
Ah yes, that has got it going!
I like how the animation goes into the Cull thread, that really helps even things out.
I am still getting some sort of deadlock in one of the levels in the game that is not happening in the other level I tested so I guess I can sort out what is causing that by the difference in the levels. Is there an easier way to find out what is causing the lock?
I thought you might be interested in the performance numbers:
Non-threaded build: 40-45 fps
Threaded build WITHOUT threading-model set: 30ish fps
Threaded build WITH threading-model set: 55-65 fps
Supposing I wanted to get the best speed on a single core or multiple cores, would I need two builds of Panda (threaded and non-threaded)?
Well, you can break into the running Panda with a debugger and see where in the code each thread is stopped; that’s what I did in this case. But that may not tell you much unless you already have a good sense of what the code is supposed to be doing. (I saw that the main thread was waiting for the cull thread to complete, which is normal, and that the cull thread was waiting to access a GeomVertexData, which isn’t normal. That gave me the clue to suspect that the main thread was holding the lock on the GeomVertexData.)
However, if you build a version of Panda with DEBUG_THREADS defined, then it will compile a special version of the threading library that monitors each lock and unlock and attempts to check for deadlock. Not all deadlock conditions can be reported, but many can (this example would have been), and if it detects the deadlock it will tell you what locks were being held by which threads, which may also give you insight. Of course the DEBUG_THREADS version of Panda runs more slowly.
Yes. The fastest possible speed on a single core is always with a single-threaded build. Having threading support available necessarily adds additional overhead to all low-level operations across the board, even if you’re not using threads at the moment.
Thanks, that was helpful. I managed to find a couple more problems related to the vertex data issue. Now it runs for a fair bit longer, but I still get a deadlock.
I tried the thread debug build, this is what I’ve got:
:thread(error):
****************************************************************
***** Deadlock detected! *****
****************************************************************
:thread(error): Thread Cull attempted to lock Mutex 03D9BDE0 which is held by MainThread Main
:thread(error): MainThread Main is blocked waiting on CyclerMutex PandaNode::CData which is held by Thread Cull
:thread(error): Deadlock!
Assertion failed: Deadlock at line 205 of c:\work\panda3d\panda\src\pipeline\mutexDebug.cxx
Assertion failed: node->_prev != NULL && node->_prev->_next == node && node->_next->_prev == node at line 92 of c:\work\panda3d\built\include\linkedListNode.I
Assertion failed: Thread Cull attempted to release Mutex 03D9BDE0 which it does not own at line 369 of c:\work\panda3d\panda\src\pipeline\mutexDebug.cxx
Traceback (most recent call last):
File "C:\work\panda3d\built\direct\showbase\ShowBase.py", line 1656, in __resetPrevTransform
PandaNode.resetAllPrevTransform()
AssertionError: Deadlock at line 205 of c:\work\panda3d\panda\src\pipeline\mutexDebug.cxx
:task(error): Exception occurred in PythonTask resetPrevTransform
Traceback (most recent call last):
File "client.py", line 564, in <module>
game.run()
File "C:\work\panda3d\built\direct\showbase\ShowBase.py", line 2648, in run
self.taskMgr.run()
File "C:\work\panda3d\built\direct\task\Task.py", line 502, in run
self.step()
File "C:\work\panda3d\built\direct\task\Task.py", line 460, in step
self.mgr.poll()
File "C:\work\panda3d\built\direct\showbase\ShowBase.py", line 1656, in __resetPrevTransform
PandaNode.resetAllPrevTransform()
AssertionError: Deadlock at line 205 of c:\work\panda3d\panda\src\pipeline\mutexDebug.cxx
CData is short for CycleData, which is the internal name for the data that is copied between pipeline stages across all Panda objects. In this case, it’s PandaNode::CData, which means the deadlock is on something in PandaNode: something in the main thread is waiting for a PandaNode lock (held by Cull) at the same time as it’s also holding this unnamed Mutex 03D9BDE0, and in the meantime Cull tried to grab this unnamed Mutex.
That’s deadlock all right, but it’s not very informative since we’re not sure what the unnamed mutex was. The stack trace gives us a bit of a clue, that it happened during resetAllPrevTransform(), but I’m still not sure what the contention is. Bleah.
After some digging, it seems this deadlock is related to something in my positional audio update code, and possibly FMOD. The more sounds I have the more quickly it will deadlock.
My guess would be as a result of many calls to sound.status(), sound.set3dAttributes(), or sound.setVolume(), but who knows.
I’ll try to isolate the sound code and see if it still has a problem on its own. That should make a fairly good demo of the problem.
Ah, OK. I’d be willing to bet that Panda’s FMod audio layer (and its OpenAL audio layer, for that matter) doesn’t properly protect itself against multithreaded access.
Both of these were originally implemented by people who likely weren’t thinking in terms of multithreaded code.
I’ll put it on the list to investigate. In the meantime, one easy way to assure yourself that it is, in fact, strictly related to the audio subsystem is to run with “audio-library-name null” in your Config.prc file.
