need advise : use of multi process in panda3d ?

PhilippeC · October 18, 2015, 7:18am

Hello

I need an advise (see questions).

Context :
In my game I use sqlite database (import sqlite3).
The game saved every 2 mn few parameters in order to avoid to the end user to do a save.
The game uses for the save a dedicated panda task (base.taskMgr.add)

Issue :
I have a issue with the game play.
When the end user move the character, the character slow down when the save of the x,y,z coordinates of the character is done in the database.

Solution:
My idea is to use the python processes and pipe (from multiprocessing import Process, Pipe)

one process for panda
one process for the database
and a pipe to communicate between the 2 processes

I have understood that the panda3D task is executed in one process, and the tasks are executed one by one between 2 frames.
My PC have 4 processors : if panda3D runs in one processor and the database access is running in an other processor, I expect no slow down during the game.

My questions :
Do I have a good understanding of the root cause of the slow down ?
Before to continue and change drastically the code I would like to know if there is no restrictions/issues with panda3d and packaging ?
Do you know if this kind of development has been done with panda3d and is successful ?

Other solutions to avoid this kind of slow down during the game are welcome.

Thanks by advance for your help

Philippe

wezu · October 18, 2015, 9:22am

I’m not sure how good Panda works with multiprocessing Process/Pipe, I never tried that, but there are a few thing I’d suggest first before going that rout.

Make the character movement speed framerate independent. Even if the rendering slows down the characters will move with the same speed… unless we are talking about a drop from 60fps to 15fps then the shuttering will be noticeable no matter what you do.

Try the threaded pipeline as described in the manual: panda3d.org/manual/index.php … r_Pipeline

Use a custom task chain:

taskMgr.setupTaskChain('sqlTaskChain', numThreads = 1, tickClock = None, frameSync = True)
taskMgr.add(self.mySqlTaskThing, 'sqlTask', taskChain = 'sqlTaskChain')

I’m not 100% sure if the parameters in setupTaskChain() are right, but it’s a start.

Solstice · October 18, 2015, 6:23pm

Why would you use a database?

I suggest looking at mmap and shared memory. It’s faster, more flexible
and you eliminite lots of unknown factors. Plus it’s really easy to handle.

Sorry if I can’t help more than that.

PhilippeC · November 3, 2015, 9:41pm

I have done this simple script in order to test the multiprocess. My game is no more slow down.

I have also found a solution to debug with pyscripter the panda3D process or the server process.

To see the 2 processes running launch the script in a cmd window.
With pyscripter you can not see the print done in the process.

import sys
import os
from multiprocessing import Process, Pipe

import time

from direct.task import Task
from pandac.PandaModules import WindowProperties
from pandac.PandaModules import loadPrcFileData
from direct.showbase.ShowBase import ShowBase

class ProcessServer(object):
    def __init__(self,oPipe):

        self.oPipe = oPipe

        self.mLaunchAction()

    def mLaunchAction(self):

        while True:
            self.checkPipe()
            time.sleep(1)

    def checkPipe(self):
        while self.oPipe.poll():
            request = self.oPipe.recv()
            if request[0] == "print":
                print "ProcessSer checkPipe", str(request)
                self.oPipe.send(["print","ProcessSer","message received"])

        return True

class Panda3dApp(ShowBase):
    def __init__(self, width, height, oPipe):
        ShowBase.__init__(self)

        self.oPipe = oPipe

        wp = WindowProperties()
        wp.setSize(width, height)
        base.openDefaultWindow(props=wp, gsg=None)

        self.loadSmiley()

        base.taskMgr.add(self.checkPipe, "check pipe")
        self.oPipe.send(["print","Panda3dApp","message send"])
        self.run()

    def loadSmiley(self):
        s = loader.loadModel("smiley.egg")
        s.reparentTo(render)
        s.setY(5)

    def checkPipe(self, task):

        while self.oPipe.poll():
            request = self.oPipe.recv()
            if request[0] == "print":
                print "Panda3dApp checkPipe", str(request)
                self.oPipe.send(["print","Panda3dApp","message received"])

        return Task.cont

# Test
if __name__ == "__main__":

    oPipeParent, oPipeChild = Pipe()

    #debug panda3d to comment if server process shall be debug
    oProcess = Process(target=ProcessServer, args=(oPipeChild,))
    oProcess.start()
    oPand3D = Panda3dApp(500, 500, oPipeParent)

    #debug process server to comment if panda3d process shall be debug
##    oPand3D = Process(target=Panda3dApp, args=(500, 500, oPipeParent))
##    oPand3D.start()
##    oProcess = ProcessServer(oPipeChild)

PhilippeC · November 3, 2015, 9:46pm

But I have an issue where i do the package with panda3d 1.9.0 : an error message is raised for multiprocess.


"C:\Program Files (x86)\Panda3D\panda3d.exe" C:\Panda3D-1.9.0-x64\packp3d1.9.p3d -o C:\Users\philippe\Documents\game\editor\test\multiprocess\mp-V7.p3d -d C:\Users\philippe\Documents\game\editor\test\multiprocess -m mp-V7.py -r morepy -D

I got the folowing error message:

:downloader: [02F02BC8] begin GET [ http://runtime.panda3d.org/morepy/cmu_1.9/win_i386/morepy.cmu_1.9.win_i386.xml ]
:downloader: [02F02BC8] begin GET [ http://runtime.panda3d.org/morepy/cmu_1.9/win_i386/morepy.cmu_1.9.win_i386.mf.pz ]
Installing Python standard library
Install complete.
:PackageInfo: Package morepy uses 17 MB
:AppRunner: Total Panda3D disk space used: 349 MB
:AppRunner: Configured max usage is: 2048 MB
Traceback (most recent call last):
  File "C:\buildslave\rtdist-windows-i386\build\built\direct\showbase\Messenger.py", line 354, in __taskChainDispatch
  File "C:\buildslave\rtdist-windows-i386\build\built\direct\showbase\Messenger.py", line 412, in __dispatch
  File "C:\buildslave\rtdist-windows-i386\build\built\direct\p3d\AppRunner.py", line 755, in __startIfReady
  File "VFSImporter", line 148, in load_module
  File "C:\Users\philippe\Documents\game\editor\test\multiprocess\mp-V7.py", line 68, in <module>

    oPipeParent, oPipeChild = Pipe()

  File "C:\thirdparty\win-python\lib\multiprocessing\__init__.py", line 107, in Pipe
  File "C:\thirdparty\win-python\lib\multiprocessing\connection.py", line 223, in Pipe
WindowsError: [Error 161] the path is not valid

PhilippeC · November 3, 2015, 9:55pm

I have tried to prepare the package with panda3d 1.8.1

I got the folwoing error message :

  File "c:\python27\lib\multiprocessing\__init__.py", line 107, in Pipe
    return Pipe(duplex)
  File "c:\python27\lib\multiprocessing\connection.py", line 222, in Pipe
    win32.PIPE_WAIT,
WindowsError: [Error 123] the syntax of the file name or the directory is incorrect
Failure on startup.

With 1.8.1 the directory used is : c:\python27\lib\multiprocessing
With 1.9.0 the directory used is : C:\thirdparty\win-python\lib\multiprocessing

rdb · November 4, 2015, 7:19am

Since the Panda3D runtime embeds the interpreter, I’m afraid there’s no way to use Python’s multiprocessing module in the runtime environment.

You could instead try to run an external .p3d file and find a way to communicate with it.

PhilippeC · November 4, 2015, 8:14pm

Ok , I understood the root cause of the issue.

thank you

philippe