Hardware instancing problem with Panda 1.8

Return to General Discussion

Hardware instancing problem with Panda 1.8

Postby Adanf » Thu Mar 01, 2012 9:44 pm

Hi guys, i am new here and I have a problem with the instantiation of objects on GPU.
I think it's an issue of the new version of Panda because I used the code in this post: http://www.panda3d.org/forums/viewtopic.php?t=8680 (the post with the code Panda3d and Cg) but when I start the program, python gives me this error:

Code: Select all
  File "test_istanze.py", line 9, in <module>
    from pandac.PandaModules import Point3, Vec4, PTAVecBase4, Shader
ImportError: cannot import name PTAVecBase4


I have noticed that this module exists in the 1.7.2 reference, but doesn't in the 1.8.
Can anyone help me to run this great feature?
Thanks to all
(sorry for my bad english :D )
User avatar
Adanf
 
Posts: 86
Joined: Tue Feb 21, 2012 7:41 pm
Location: Italy

Postby teedee » Fri Mar 02, 2012 2:21 am

For 1.8, replace PTAVecBase4 with PTA_LVecBase4f and instead of filling it with Vec4 use UnalignedLVecBase4f.
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby Adanf » Sat Mar 03, 2012 4:46 am

but k must be a power of 2?
I set k to 100 and gives me this error:
Code: Select all
:display:gsg:glgsg(error): offsets: incorrect number of elements, expected 1024 got 400


I changed the code like this (this is a code snippet):
Code: Select all
      self.cube=self.loader.loadModel(pandafile)
      self.cube.flattenLight()
      self.cube.reparentTo(render)

      k = ABSPOS**2
      offsets = PTA_LVecBase4f.emptyArray(k);
      count = 0
      for x in range(ABSPOS):
         xp=x*10
         for y in range(ABSPOS):
            offsets[count] = UnalignedLVecBase4f(xp,y*10, 0, 0)
            count += 1
      self.cube.setShaderInput('offsets', offsets)
      self.cube.setShader(Shader.load('instance.cg'))
      self.cube.setInstanceCount(k)


thanks for all
User avatar
Adanf
 
Posts: 86
Joined: Tue Feb 21, 2012 7:41 pm
Location: Italy

Postby teedee » Sat Mar 03, 2012 6:27 pm

It can be any number, but the number of elements (k) in the array must match the number of instances specified in the cg shader.

In the cg file:
Code: Select all
uniform float4 offsets[256],
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby Adanf » Mon Mar 19, 2012 6:04 pm

Excuse me if I resume this old post, but I have another problem.
Over a number of instances (for example k = 10000) the shader gets this error:
Code: Select all
:gobj(error): instance.cg: (7) : error C5108: unknown semantics "INSTANCEID" specified for "l_id"

I have set
Code: Select all
 offsets = PTA_LVecBase4f.emptyArray(10000)

and
Code: Select all
uniform float4 offsets[10000]


Is not there a limit on instances managed by a single call?
Thanks to all.
User avatar
Adanf
 
Posts: 86
Joined: Tue Feb 21, 2012 7:41 pm
Location: Italy

Postby teedee » Tue Mar 20, 2012 3:38 am

I have not tried before with so many instances.
I suppose if you found (as an example) that the limit was 5000 and you needed 10000 instances, you could make two of the same object with 5000 instances each.
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby rdb » Tue Mar 20, 2012 12:13 pm

When you get that error, it usually means that it switched to a different profile, one that doesn't support instancing. Perhaps the change in array size triggered this. Try manually setting a profile (using a //Cg profile line) to one that you know will work.
rdb
 
Posts: 9590
Joined: Mon Dec 04, 2006 5:58 am
Location: Netherlands

Postby Adanf » Wed Mar 21, 2012 1:00 pm

rdb wrote:When you get that error, it usually means that it switched to a different profile, one that doesn't support instancing. Perhaps the change in array size triggered this. Try manually setting a profile (using a //Cg profile line) to one that you know will work.


I do not know any type of cg profile, you can post me an example? (i'm newbie in shading :D )
thanks
User avatar
Adanf
 
Posts: 86
Joined: Tue Feb 21, 2012 7:41 pm
Location: Italy

Postby rdb » Wed Mar 21, 2012 1:08 pm

There's plenty of information on Cg on the internet.
http://http.developer.nvidia.com/Cg/index_profiles.html
rdb
 
Posts: 9590
Joined: Mon Dec 04, 2006 5:58 am
Location: Netherlands

Postby Badger » Sun Mar 25, 2012 5:54 am

Hello

I'm not sure this is the problem.

I've been trying to get the project below working in 1.8.0, by changing the old type as described above, yet i get the same error on the schader with INSTANCEID.

http://www.etc.cmu.edu/projects/pandase ... ancing.rar

edit: I guess this has to do with this project not being compatible with ATI cards...

thanks
Badger
Badger
 
Posts: 19
Joined: Wed Nov 18, 2009 4:25 am

Postby Badger » Sat Mar 31, 2012 9:57 am

Hi there,

I've altered the example linked above for Panda3D 1.8.0, as show below (i didn't include it as code, as that doesn't work with the color tags apparently):

k = 1000
input = PTA_LVecBase4f.emptyArray(k);
count = 0
for i in range(10):
for j in range(k/10):
input[count] = UnalignedLVecBase4f(i*3,j*-8,0,0)
count += 1



lvec = PTA_LVecBase4f([UnalignedLVecBase4f(-4,0,0,0),UnalignedLVecBase4f(4,0,0,0)]);
self.ralph.setShaderInput("light", plnp)
self.ralph.setShaderInput("fIN.Kd", PTAFloat([1.4]))
self.ralph.setShaderInput("fIN.Ka", PTAFloat([0.2]))
self.ralph.setShaderInput("Ly", input)
self.ralph.setShaderInput("instances_position", lvec)
#self.ralph.setShader(Shader.load("shader.c"))
self.ralph.setShader(Shader.load(Shader.SLGLSL,"hgi_glsl_v.sha","hgi_glsl_f.sha"))

self.ralph.setInstanceCount(k)


This however throws an InstanceID semantics error when compiling (?) the shader. So only 1 ralph is showing.

http://www.panda3d.org/forums/viewtopic.php?t=10816

In this post, i found GLSL shaders which i adapted as below (only hgi_glsl_v.sha, didn't touch the fragment shader):

//GLSL

uniform vec4 Ly[1000];

varying vec3 vertex_light_position;
varying vec3 vertex_light_half_vector;
varying vec3 vertex_normal;

void main(void)
{
vec4 vpos= gl_Vertex + Ly[gl_InstanceID];
gl_Position = gl_ModelViewProjectionMatrix * vpos;

vertex_normal = normalize(gl_NormalMatrix * gl_Normal);
vertex_light_position = normalize(gl_LightSource[0].position.xyz);
vertex_light_half_vector = normalize(gl_LightSource[0].halfVector.xyz);
gl_FrontColor = gl_Color;

gl_TexCoord[0] = gl_MultiTexCoord0;
}


I've looked up glShaderContext_src.cxx of the Panda 1.8.0 source code & compared it to the patch included in this post http://www.panda3d.org/forums/viewtopic.php?t=10816 & i think it is already included.

When i run the project, the shader compiles correctly, yet no raplhs are shown.

Any clues what i'm doing wrong? Or some good pointers to GLSL programming info?

Thanks in advance
Badger
 
Posts: 19
Joined: Wed Nov 18, 2009 4:25 am

Postby rdb » Sun Apr 01, 2012 1:00 am

I don't think that passing arrays to GLSL shaders is currently supported, sorry.
rdb
 
Posts: 9590
Joined: Mon Dec 04, 2006 5:58 am
Location: Netherlands

Postby teedee » Sun Apr 01, 2012 3:32 am

I trimmed down my own shader which does work, give it a try. It uses a 4x4 matrix for each instance stacked up in a vec4 array. This is because Panda doesn't support arrays of matrices in GLSL yet, but vec4 it does. You can simplify if you only need position and not rotation.

Vertex shader:
Code: Select all
//GLSL
#version 140
#extension GL_ARB_compatibility : enable

const int num_instances = 1000;
uniform vec4 shader_data[num_instances * 4];

void main() {
  mat4 transform = mat4(shader_data[gl_InstanceID * 4], shader_data[gl_InstanceID * 4 + 1], shader_data[gl_InstanceID * 4 + 2], shader_data[gl_InstanceID * 4 + 3]);
  gl_Position = gl_ModelViewProjectionMatrix * (gl_Vertex * transform);
  gl_TexCoord[0] = gl_MultiTexCoord0;
  gl_FrontColor = gl_Color;
}

Fragment shader:
Code: Select all
//GLSL
#version 140
#extension GL_ARB_compatibility : enable

uniform sampler2D p3d_Texture0;

void main() {
  gl_FragColor = texture(p3d_Texture0, vec2(gl_TexCoord[0]));
}
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby Badger » Mon Apr 02, 2012 2:03 pm

Thank you, i think i can just grasp the concept :shock:

I have an array of nodepath (soldiers[n]).
I loop trough & get the tranformation matrix of each nodepath.
Then I assign the components of this matrix columnwise to a shader_data Vec4 array:

Code: Select all
soldiertrans = self.soldiers[n].getMat()

shader_data.append(soldiertrans.getCol(0))
shader_data.append(soldiertrans.getCol(1))
shader_data.append(soldiertrans.getCol(2))
shader_data.append(soldiertrans.getCol(3)) 


Then I send this to the shader
Code: Select all
soldier_master.setShaderInput("tmat",shader_data);


Everthing compiles, yet no ralphs are visible :(

Did I do anything wrong with the above?

(The ARB_GL_compatibility throws a warning, i commented it out, no change).

Thanks
Badger
 
Posts: 19
Joined: Wed Nov 18, 2009 4:25 am

Postby teedee » Mon Apr 02, 2012 2:43 pm

You might need to do something like
Code: Select all
        for i in range(4):
            col = soldiertrans.getCol(i)
            shader_data.append(UnalignedLVecBase4f(col[0], col[1], col[2], col[3]))

because getCol alone will just give a regular vec4.

I have some code that I should be able to extract out of my game that I think will be helpful, I can probably find some time tonight to have a look.
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby teedee » Tue Apr 03, 2012 3:28 am

Here is my instancing code. I think it would make a good starting point for an easier interface to include in Panda but I really don't have the time to bring it to completion.
This actually allows you to have a varying number of instances, so you can add or remove instances as you go. This is done in the vertex shader I posted earlier by modifying one of the lines like this:
Code: Select all
const int num_instances = %i;

The code creates many shaders with different numbers substituted for %i and applies the right shader as needed (default max 1024, but you can increase that in the code).
You could put in a new module, say, instancemanager.py:
Code: Select all
from panda3d.core import *


class InstanceManager(object):
    def __init__(self, parent=render, max_instances=1024):
        self.parent = parent
        self.no_render = NodePath('no_render')
        self.categories = {}
        v = open('shaders/instance_v.glsl').read()
        f = open('shaders/instance_f.glsl').read()
        self.instance_shader = [Shader.make(Shader.SLGLSL, v % i, f) for i in range(max_instances + 1)]
        taskMgr.add(self.update_category_shaders, 'instance manager')

    def add(self, model):
        try:
            category = self.categories[model]
        except KeyError:
            category = InstanceCategory(self, model)
        return Instance(category)

    def update_category_shaders(self, task):
        for category in self.categories:
            self.categories[category].update_shader()
        return task.cont


class InstanceCategory(object):
    def __init__(self, manager, model):
        self.manager = manager
        self.model = loader.loadModel(model)
        bs = OmniBoundingVolume()
        self.model.node().setBounds(bs)
        self.model.node().setFinal(1)
        self.model.reparentTo(manager.no_render)
        self.instances = []
        self.shader_data = PTA_LVecBase4f()
        self.need_update = True
        self.manager.categories[model] = self

    def add(self):
        count = len(self.instances)
        if count == 1:
            self.model.reparentTo(self.manager.parent)
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.model.setShader(self.manager.instance_shader[count])
        self.model.setInstanceCount(count)
        self.update_shader()

    def remove(self, index):
        if len(self.instances) > 1:
            self.shader_data[index * 4] = self.shader_data[-4]
            self.shader_data[index * 4 + 1] = self.shader_data[-3]
            self.shader_data[index * 4 + 2] = self.shader_data[-2]
            self.shader_data[index * 4 + 3] = self.shader_data[-1]
            self.instances[index] = self.instances[-1]
            self.instances[index].id = index
        else:
            self.model.reparentTo(self.manager.no_render)
        for i in range(4):
            self.shader_data.popBack()
        self.instances.pop()
        count = len(self.instances)
        self.model.setShader(self.manager.instance_shader[count])
        self.model.setInstanceCount(count)

    def update_shader(self):
        if self.need_update:
            self.model.setShaderInput('shader_data[0]', self.shader_data)
            self.model.setShaderInput('shader_data', self.shader_data)
            self.need_update = False


class Instance(object):
    def __init__(self, category):
        self.category = category
        self.id = len(category.instances)
        category.instances.append(self)
        category.add()

    def destroy(self):
        self.category.remove(self.id)

    def update_transform_inputs(self, mat):
        for i in range(4):
            col = mat.getCol(i)
            self.category.shader_data[self.id * 4 + i] = UnalignedLVecBase4f(col[0], col[1], col[2], col[3])
        self.category.need_update = True

You would use it in a program by doing something like this:
Code: Select all
from instancemanager import InstanceManager
instances = InstanceManager()

class FunBox(object):
  def __init__(self, pos):
    self.instance = instances.add('models/box')
    self.np = NodePath('box')
    self.np.setPos(pos)
    self.instance.update_transform_inputs(self.np.getMat())

  def destroy(self):
    self.instance.destroy()
    self.np.removeNode()


# make some instances
box_a = FunBox(Point3(0, 10, 0))
box_b = FunBox(Point3(0, 15, 0))
box_c = FunBox(Point3(0, 20, 0))
box_d = FunBox(Point3(0, 25, 0))

box_c.destroy() # get rid of one of them

The way it is set up right now it relies on you to call update_transform_inputs on your Instance each time you need to move it, but you could automate this with a bit of work to the InstanceManager. I leave it as an exercise to the reader.
I did not actually test this version with my game-specific stuff stripped out, but feel free to reply with any questions. I've been meaning to share this with those strugging with Panda's recently added instancing functionality but have been super busy as of late.

Now that I look through the code again, what you might be missing is the setShaderInput call which I have two of in this code:
Code: Select all
self.model.setShaderInput('shader_data[0]', self.shader_data)
self.model.setShaderInput('shader_data', self.shader_data)

Some video card drivers want the input name as usual, and some want it with [0] put on the end. Both are valid, but each driver could accept one or the other or both, so best to be safe and have both lines.
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby Badger » Tue Apr 03, 2012 2:33 pm

Thank you very much, you filled my need before I even realised I needed it already.

I'm starting to suspect my system is not up to geometry instancing, as I'm pretty sure I've got it right & still I get no instances rendered...

Could someone please check the project below to see if you see pawns in alongside the road ralph is marching on?

http://www.badger1815.be/examples/Geometry%20Instancing.rar

regards
Badger
Badger
 
Posts: 19
Joined: Wed Nov 18, 2009 4:25 am

Postby teedee » Tue Apr 03, 2012 4:18 pm

Yes I see 4 pawns.
As far as I've determined, the minimums for hardware instancing are:
NVIDIA - Geforce 8000-series or newer
ATI/AMD - any Radeon HD card
Intel - not sure, but it works on the HD 3000 in my laptop
teedee
 
Posts: 861
Joined: Tue May 12, 2009 11:33 pm
Location: Kepler-22b

Postby Badger » Tue Apr 03, 2012 4:26 pm

I guess that is kind of good news.

Mine's a ATI Radeon HD 6900, should be enough. I've checked the ATI tray tools & geometry instancing is enabled.

I'll try & reinstall all drivers first then.

cheers
B
Badger
 
Posts: 19
Joined: Wed Nov 18, 2009 4:25 am


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 0 guests