Hardware instancing problem with Panda 1.8

[Adanf]

Hi guys, i am new here and I have a problem with the instantiation of objects on GPU.
I think it's an issue of the new version of Panda because I used the code in this post: http://www.panda3d.org/forums/viewtopic.php?t=8680 (the post with the code Panda3d and Cg) but when I start the program, python gives me this error:

Code: Select all

  File "test_istanze.py", line 9, in <module>
    from pandac.PandaModules import Point3, Vec4, PTAVecBase4, Shader
ImportError: cannot import name PTAVecBase4


I have noticed that this module exists in the 1.7.2 reference, but doesn't in the 1.8.
Can anyone help me to run this great feature?
Thanks to all
(sorry for my bad english )

[teedee]

For 1.8, replace PTAVecBase4 with PTA_LVecBase4f and instead of filling it with Vec4 use UnalignedLVecBase4f.

[Adanf]

but k must be a power of 2?
I set k to 100 and gives me this error:

Code: Select all

:display:gsg:glgsg(error): offsets: incorrect number of elements, expected 1024 got 400


I changed the code like this (this is a code snippet):

Code: Select all

      self.cube=self.loader.loadModel(pandafile)
      self.cube.flattenLight()
      self.cube.reparentTo(render)

      k = ABSPOS**2
      offsets = PTA_LVecBase4f.emptyArray(k);
      count = 0
      for x in range(ABSPOS):
         xp=x*10
         for y in range(ABSPOS):
            offsets[count] = UnalignedLVecBase4f(xp,y*10, 0, 0)
            count += 1
      self.cube.setShaderInput('offsets', offsets)
      self.cube.setShader(Shader.load('instance.cg'))
      self.cube.setInstanceCount(k)


thanks for all

[teedee]

It can be any number, but the number of elements (k) in the array must match the number of instances specified in the cg shader.

In the cg file:

Code: Select all

uniform float4 offsets[256],

[Adanf]

Excuse me if I resume this old post, but I have another problem.
Over a number of instances (for example k = 10000) the shader gets this error:

Code: Select all

:gobj(error): instance.cg: (7) : error C5108: unknown semantics "INSTANCEID" specified for "l_id"


I have set

Code: Select all

 offsets = PTA_LVecBase4f.emptyArray(10000)

and

Code: Select all

uniform float4 offsets[10000]

Is not there a limit on instances managed by a single call?
Thanks to all.

[teedee]

I have not tried before with so many instances.
I suppose if you found (as an example) that the limit was 5000 and you needed 10000 instances, you could make two of the same object with 5000 instances each.

[rdb]

When you get that error, it usually means that it switched to a different profile, one that doesn't support instancing. Perhaps the change in array size triggered this. Try manually setting a profile (using a //Cg profile line) to one that you know will work.

[Adanf]

When you get that error, it usually means that it switched to a different profile, one that doesn't support instancing. Perhaps the change in array size triggered this. Try manually setting a profile (using a //Cg profile line) to one that you know will work.

I do not know any type of cg profile, you can post me an example? (i'm newbie in shading )
thanks

[rdb]

There's plenty of information on Cg on the internet.
http://http.developer.nvidia.com/Cg/index_profiles.html

[Badger]

Hello

I'm not sure this is the problem.

I've been trying to get the project below working in 1.8.0, by changing the old type as described above, yet i get the same error on the schader with INSTANCEID.

http://www.etc.cmu.edu/projects/pandase ... ancing.rar

edit: I guess this has to do with this project not being compatible with ATI cards...

thanks
Badger

[Badger]

Hi there,

I've altered the example linked above for Panda3D 1.8.0, as show below (i didn't include it as code, as that doesn't work with the color tags apparently):

k = 1000
input = PTA_LVecBase4f.emptyArray(k);
count = 0
for i in range(10):
for j in range(k/10):
input[count] = UnalignedLVecBase4f(i*3,j*-8,0,0)
count += 1



lvec = PTA_LVecBase4f([UnalignedLVecBase4f(-4,0,0,0),UnalignedLVecBase4f(4,0,0,0)]);
self.ralph.setShaderInput("light", plnp)
self.ralph.setShaderInput("fIN.Kd", PTAFloat([1.4]))
self.ralph.setShaderInput("fIN.Ka", PTAFloat([0.2]))
self.ralph.setShaderInput("Ly", input)
self.ralph.setShaderInput("instances_position", lvec)
#self.ralph.setShader(Shader.load("shader.c"))
self.ralph.setShader(Shader.load(Shader.SLGLSL,"hgi_glsl_v.sha","hgi_glsl_f.sha"))

self.ralph.setInstanceCount(k)

This however throws an InstanceID semantics error when compiling (?) the shader. So only 1 ralph is showing.

http://www.panda3d.org/forums/viewtopic.php?t=10816

In this post, i found GLSL shaders which i adapted as below (only hgi_glsl_v.sha, didn't touch the fragment shader):

//GLSL

uniform vec4 Ly[1000];

varying vec3 vertex_light_position;
varying vec3 vertex_light_half_vector;
varying vec3 vertex_normal;

void main(void)
{
vec4 vpos= gl_Vertex + Ly[gl_InstanceID];
gl_Position = gl_ModelViewProjectionMatrix * vpos;

vertex_normal = normalize(gl_NormalMatrix * gl_Normal);
vertex_light_position = normalize(gl_LightSource[0].position.xyz);
vertex_light_half_vector = normalize(gl_LightSource[0].halfVector.xyz);
gl_FrontColor = gl_Color;

gl_TexCoord[0] = gl_MultiTexCoord0;
}


I've looked up glShaderContext_src.cxx of the Panda 1.8.0 source code & compared it to the patch included in this post http://www.panda3d.org/forums/viewtopic.php?t=10816 & i think it is already included.

When i run the project, the shader compiles correctly, yet no raplhs are shown.

Any clues what i'm doing wrong? Or some good pointers to GLSL programming info?

Thanks in advance

[rdb]

I don't think that passing arrays to GLSL shaders is currently supported, sorry.

[teedee]

I trimmed down my own shader which does work, give it a try. It uses a 4x4 matrix for each instance stacked up in a vec4 array. This is because Panda doesn't support arrays of matrices in GLSL yet, but vec4 it does. You can simplify if you only need position and not rotation.

Vertex shader:

Code: Select all

//GLSL
#version 140
#extension GL_ARB_compatibility : enable

const int num_instances = 1000;
uniform vec4 shader_data[num_instances * 4];

void main() {
  mat4 transform = mat4(shader_data[gl_InstanceID * 4], shader_data[gl_InstanceID * 4 + 1], shader_data[gl_InstanceID * 4 + 2], shader_data[gl_InstanceID * 4 + 3]);
  gl_Position = gl_ModelViewProjectionMatrix * (gl_Vertex * transform);
  gl_TexCoord[0] = gl_MultiTexCoord0;
  gl_FrontColor = gl_Color;
}

Fragment shader:

Code: Select all

//GLSL
#version 140
#extension GL_ARB_compatibility : enable

uniform sampler2D p3d_Texture0;

void main() {
  gl_FragColor = texture(p3d_Texture0, vec2(gl_TexCoord[0]));
}

[Badger]

Thank you, i think i can just grasp the concept

I have an array of nodepath (soldiers[n]).
I loop trough & get the tranformation matrix of each nodepath.
Then I assign the components of this matrix columnwise to a shader_data Vec4 array:

Code: Select all

soldiertrans = self.soldiers[n].getMat()

shader_data.append(soldiertrans.getCol(0))
shader_data.append(soldiertrans.getCol(1))
shader_data.append(soldiertrans.getCol(2))
shader_data.append(soldiertrans.getCol(3)) 

Then I send this to the shader

Code: Select all

soldier_master.setShaderInput("tmat",shader_data);

Everthing compiles, yet no ralphs are visible

Did I do anything wrong with the above?

(The ARB_GL_compatibility throws a warning, i commented it out, no change).

Thanks

[teedee]

You might need to do something like

Code: Select all

        for i in range(4):
            col = soldiertrans.getCol(i)
            shader_data.append(UnalignedLVecBase4f(col[0], col[1], col[2], col[3]))

because getCol alone will just give a regular vec4.

I have some code that I should be able to extract out of my game that I think will be helpful, I can probably find some time tonight to have a look.

[teedee]

Here is my instancing code. I think it would make a good starting point for an easier interface to include in Panda but I really don't have the time to bring it to completion.
This actually allows you to have a varying number of instances, so you can add or remove instances as you go. This is done in the vertex shader I posted earlier by modifying one of the lines like this:

Code: Select all

const int num_instances = %i;

The code creates many shaders with different numbers substituted for %i and applies the right shader as needed (default max 1024, but you can increase that in the code).
You could put in a new module, say, instancemanager.py:

Code: Select all

from panda3d.core import *


class InstanceManager(object):
    def __init__(self, parent=render, max_instances=1024):
        self.parent = parent
        self.no_render = NodePath('no_render')
        self.categories = {}
        v = open('shaders/instance_v.glsl').read()
        f = open('shaders/instance_f.glsl').read()
        self.instance_shader = [Shader.make(Shader.SLGLSL, v % i, f) for i in range(max_instances + 1)]
        taskMgr.add(self.update_category_shaders, 'instance manager')

    def add(self, model):
        try:
            category = self.categories[model]
        except KeyError:
            category = InstanceCategory(self, model)
        return Instance(category)

    def update_category_shaders(self, task):
        for category in self.categories:
            self.categories[category].update_shader()
        return task.cont


class InstanceCategory(object):
    def __init__(self, manager, model):
        self.manager = manager
        self.model = loader.loadModel(model)
        bs = OmniBoundingVolume()
        self.model.node().setBounds(bs)
        self.model.node().setFinal(1)
        self.model.reparentTo(manager.no_render)
        self.instances = []
        self.shader_data = PTA_LVecBase4f()
        self.need_update = True
        self.manager.categories[model] = self

    def add(self):
        count = len(self.instances)
        if count == 1:
            self.model.reparentTo(self.manager.parent)
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.shader_data.pushBack(UnalignedLVecBase4f())
        self.model.setShader(self.manager.instance_shader[count])
        self.model.setInstanceCount(count)
        self.update_shader()

    def remove(self, index):
        if len(self.instances) > 1:
            self.shader_data[index * 4] = self.shader_data[-4]
            self.shader_data[index * 4 + 1] = self.shader_data[-3]
            self.shader_data[index * 4 + 2] = self.shader_data[-2]
            self.shader_data[index * 4 + 3] = self.shader_data[-1]
            self.instances[index] = self.instances[-1]
            self.instances[index].id = index
        else:
            self.model.reparentTo(self.manager.no_render)
        for i in range(4):
            self.shader_data.popBack()
        self.instances.pop()
        count = len(self.instances)
        self.model.setShader(self.manager.instance_shader[count])
        self.model.setInstanceCount(count)

    def update_shader(self):
        if self.need_update:
            self.model.setShaderInput('shader_data[0]', self.shader_data)
            self.model.setShaderInput('shader_data', self.shader_data)
            self.need_update = False


class Instance(object):
    def __init__(self, category):
        self.category = category
        self.id = len(category.instances)
        category.instances.append(self)
        category.add()

    def destroy(self):
        self.category.remove(self.id)

    def update_transform_inputs(self, mat):
        for i in range(4):
            col = mat.getCol(i)
            self.category.shader_data[self.id * 4 + i] = UnalignedLVecBase4f(col[0], col[1], col[2], col[3])
        self.category.need_update = True

You would use it in a program by doing something like this:

Code: Select all

from instancemanager import InstanceManager
instances = InstanceManager()

class FunBox(object):
  def __init__(self, pos):
    self.instance = instances.add('models/box')
    self.np = NodePath('box')
    self.np.setPos(pos)
    self.instance.update_transform_inputs(self.np.getMat())

  def destroy(self):
    self.instance.destroy()
    self.np.removeNode()


# make some instances
box_a = FunBox(Point3(0, 10, 0))
box_b = FunBox(Point3(0, 15, 0))
box_c = FunBox(Point3(0, 20, 0))
box_d = FunBox(Point3(0, 25, 0))

box_c.destroy() # get rid of one of them

The way it is set up right now it relies on you to call update_transform_inputs on your Instance each time you need to move it, but you could automate this with a bit of work to the InstanceManager. I leave it as an exercise to the reader.
I did not actually test this version with my game-specific stuff stripped out, but feel free to reply with any questions. I've been meaning to share this with those strugging with Panda's recently added instancing functionality but have been super busy as of late.

Now that I look through the code again, what you might be missing is the setShaderInput call which I have two of in this code:

Code: Select all

self.model.setShaderInput('shader_data[0]', self.shader_data)
self.model.setShaderInput('shader_data', self.shader_data)

Some video card drivers want the input name as usual, and some want it with [0] put on the end. Both are valid, but each driver could accept one or the other or both, so best to be safe and have both lines.

[Badger]

Thank you very much, you filled my need before I even realised I needed it already.

I'm starting to suspect my system is not up to geometry instancing, as I'm pretty sure I've got it right & still I get no instances rendered...

Could someone please check the project below to see if you see pawns in alongside the road ralph is marching on?

http://www.badger1815.be/examples/Geometry%20Instancing.rar

regards
Badger

[teedee]

Yes I see 4 pawns.
As far as I've determined, the minimums for hardware instancing are:
NVIDIA - Geforce 8000-series or newer
ATI/AMD - any Radeon HD card
Intel - not sure, but it works on the HD 3000 in my laptop

[Badger]

I guess that is kind of good news.

Mine's a ATI Radeon HD 6900, should be enough. I've checked the ATI tray tools & geometry instancing is enabled.

I'll try & reinstall all drivers first then.

cheers
B