Auto Deferred Shader

Return to Panda Features in Development

Auto Deferred Shader

Postby wezu » Sat Jan 14, 2017 10:36 am

During the last weekend or two I've been making something that was meant as a drop-in replacement for the default, build in shader generator.... that's not actually generating any shader code (maybe apart from some tiny configuration header) but just works and is just as easy to use as render.setShaderAuto()

The idea was, that you import a class, init it, maybe give it some optional configuration values depending on what features you'd want, and that's it. The class would automagically make a new node available anywhere (just like 'render') and all you have to do is reparent your models to this special node, even if your model has no normal map, no tangent or binormals, no gloss, glow, height map - it will just work.

That's the goal, it's not done yet, but you can try what's there if you grab the code here:
https://github.com/wezu/p3d_auto_deferred_shader

Features:
-point lights
-directional light
-spotlights
-cube map shadows for point lights
-projected shadows for spotlights (not done!)
-fog/dof
-parallax mapping
-screen space reflections
-bloom/glow
-ambient occlusion
-lut color mapping
-dithering
-fxaa

Screen from current beta:
(click for full size)
Image
https://s27.postimg.org/z8uxj1xer/def_ren_beta.png

Bad quality video:
https://youtu.be/PyOgZHCBGD4

This is in no way a competing project to the Render Pipeline Tobias is making. He is making a modern, advanced rendered with fancy things targeting OpenGL 4.x with bits and pieces of C++. Mine is closer to the firefly demo, it's all Python (+GLSL) and should run on OpenGL 3.x.

Credits:
-Josh Yelon for the original FireflyDemo (..and whoever else edited it?)
-tobspr for the color LUT, normals packing and probably something else that I can't remember
-rdb for helping with lighting, shadows and a ton of other things
-ninth for ssr
-Nvidia for FXAA(?)
-anyone else I forgot :mrgreen:
Last edited by wezu on Sun Mar 12, 2017 12:24 pm, edited 2 times in total.
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby wezu » Sun Jan 15, 2017 12:28 pm

So I've spend the last 4 hours or so trying to add spotlights... with limited success :oops:

The lighting part of spotlights work (diffuse, specular, distance and cosin falloff, volume mesh... all of it), the shadows don't work and I need help with that.

If you know anything about shadows, maths, glsl, shaders, or anything else related pleas take a moment to take a look at my shaders and help me fix them.

Here's a convenient link to the faulty shader: https://github.com/wezu/p3d_auto_deferr ... ght_f.glsl

Image
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby wezu » Sat Jan 21, 2017 12:17 pm

Bad news is that shadows are still broken ... and I think the shadows for the point/sphere lights are also kind of bad :cry:

Other bad news - you won't be able to just import from deferred_render init it and be done, I've changed the light API, it's not a 1:1 clone of the build in Panda3D Lights, but for my defence I'd like to say that I think I made it simpler.

Comparison:
Panda3D PointLight
Code: Select all
plight = PointLight('plight')
plight.setColor(VBase4(0.2, 0.2, 0.2, 1))
plight.setShadowCaster(True, 512, 512)
plight.setAttenuation((1, 0, 1)) #how far will the light go...?
plnp = render.attachNewNode(plight)
plnp.setPos(10, 20, 0)
render.setLight(plnp)

Auto Deferred Shader SphereLight
Code: Select all
self.plight=SphereLight(color=(0.2, 0.2, 0.2), pos=(10, 20, 0), radius=30.0, shadow_size=512)

Notice I put 'self' here, that's because when the SphereLight goes out of scope it will get garbage collected and the light will vanish from the scene. That's intended because now you can remove lights in a more pythonic way
Removing Panda3D PointLight
Code: Select all
if plight.isShadowCaster():
    #this should work, but I say it's not working or not working always
    #plight.setShadowCaster(False)
    buff=plight.getShadowBuffer(base.win.getGsg())
    buff.clearRenderTextures()
    base.win.getGsg().getEngine().removeWindow(buff)
    plight.setShadowCaster(False)
render.setLightOff(plnp)#do I need this? better be safe...    
plnp.removeNode()

Removing a Auto Deferred Shader SphereLight
Code: Select all
del self.plight


There's SphereLight - that's a point light, there's ConeLight - that's a spotlight and there's SceneLight - that's a directional light (at the moment there's just one per scene, the shader is ready to render more, but the interface is not).

Good news - em.. fog and depth of field is working. I've also made some presets so you don't need to do all the configuring by hand (but you also can), just tell the renderer to use either 'full', 'medium' or 'minimal' oh, and also, you don't need to keep a reference to the renderer, it installs itself in the buildins
Code: Select all
DeferredRenderer(preset='full')

There are also some ways to configure shader inputs, reload shaders and put shader pre-processor values but I can't say all that is stable API, so if someone is interested in that take a look at the code or ask me here or on IRC.
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby wezu » Thu Jan 26, 2017 10:09 am

I've put some basic informations in the wiki page of the project:
https://github.com/wezu/p3d_auto_deferr ... ki/Read-me

Things that still need to be done:
-shadows
-more docs

I'll write the docs myself, bu any help with the shadows? ... anyone? No? :cry:
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby wezu » Thu Feb 02, 2017 5:19 am

I just wanted to let everyone interested (if such people exist...) that the problems with shadows I had is now fixed, soft shadows are now working.
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby firefly » Tue Aug 08, 2017 6:57 am

Hi,

I'm trying your deferred rendering solution. I tried the demo as included in your installation, but I get horizontal bars artifacts and warnings:
"global variable gl_FragColor is deprecated after version 120"

I am using the Panda SDK of august 7 2017 (most recent with python 3.6) and am running on a GTX770 GPU with windows 7 PRO 64 bits.

deferred_shader.jpg
deferred_shader.jpg (160.54 KiB) Viewed 1196 times


Do you have any idea what could cause this?
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm

Re: Auto Deferred Shader

Postby wezu » Tue Aug 08, 2017 1:09 pm

That is just a warning and not the cause of the bars, I've got some fixes and improvements waiting on my disk but none of them solve the bar problem and as always during summer I can't get anything done.

The problem is with the AO, I've got a topic here somewhere on it, but so far no help, idea or solution, I may need to take a look at how p3d is doing AO and do a Cg->GLSL translation... unless some has a working AO shader and is willing to share (I'm not going to steal AO from the Render Pipline, not for lack of trying, can't find a thing there).
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby firefly » Wed Aug 09, 2017 5:06 am

Yes, when I switch AO off, it is gone. In your demo, the SphereLight does cast shadows, however, I can't get shadows with the SceneLight, for example:
self.light_0 = SceneLight(color=(0.8, 0.8, 0.8),direction = (-0.5, 0.5.1.0), shadow_size=512)

As far as I could tell, shadows are implemented with SceneLight, its a directional light, but I can't get it to work.

Another thing: the area that is displayed is very small. If I try to visualize an other database, the area rendered is also very small. I can't find any setNearFar statements where this can be changed. Do you know how to do that?
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm

Re: Auto Deferred Shader

Postby wezu » Wed Aug 09, 2017 5:39 am

I didn't add shadows to the scene(directional) lights. I'm not sure how to setup cascade shadow mapping and anything else sucks, even with cascade mapping you get artifacts.

The near-far frustrum is set so low to maximize depth resolution, you can move the far plane 10x or 100x further but I'd leave the near plane where it is. I didn't put any function to set this, you'll need to edit deferred_render.py, line 440(or near it)
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby firefly » Thu Aug 17, 2017 2:43 am

When I load a database that I prepared instead of your demo world, textures are not rendered. I have sent a PM to you with a link to the database, can you have alook at it to see what I did wrong? The egg file runs normally in pview and blender. I really would like to use your solution, it looks really good and runs with a high framerate, which is very important for my application.
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm

Re: Auto Deferred Shader

Postby firefly » Thu Aug 17, 2017 5:53 am

Its something in the loader. When all textures are read, only the first <Scalar> envtype { modulate } texture is assigned to a texture stage (default). All normal textures envtype <normal } are assigned to a separate texture stage. When, after the first { modulate } texture, other modulate textures are read they aren't assigned to a texture stage.
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm

Re: Auto Deferred Shader

Postby wezu » Fri Aug 18, 2017 7:01 am

One object can have one color, one normal and one height-gloss-glow-cutout texture, one object should probably also be one egg/bam file, but I think it should be possible to parse a multi-object model and set the inputs on each sub node.
The root of the problem is the way panda sets texture inputs, there's no way to know if p3d_texture1 is a color, normal, or some special map in the shader.
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby wezu » Sat Aug 19, 2017 9:06 am

firefly -> I've put some fixes on the github repo, can you try and see if it works now?
I got the sample you send me running, but with very bad performance, I had to disable shadows to get it over 20-ish fps, I'm not sure if the shadow camera respects the LODs and it's own frustum :?
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby firefly » Sat Aug 19, 2017 12:53 pm

I just downloaded your latest version, but still no textures at all. I tried full and minimal, but no textures are displayed whith the same test world as I sent you. I am using the panda version of august 7 2017, python 3.6 build.
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm

Re: Auto Deferred Shader

Postby wezu » Sat Aug 19, 2017 1:24 pm

My bad, forgot to change one function call, you can fix that yourself -line 920, replace self.setTextureInputs(model) with self.setTextureInputsRecursive(model) - or wait until I make another commit (maybe till Monday evening)
I may be totally wrong, cause I'm a dancin' fool.

NameError: global name 'elf' is not defined
User avatar
wezu
 
Posts: 1090
Joined: Tue May 19, 2009 1:03 pm

Re: Auto Deferred Shader

Postby firefly » Mon Aug 21, 2017 3:28 am

Yes, that worked perfectly. Thank you.
firefly
 
Posts: 113
Joined: Mon Apr 25, 2011 12:44 pm


Return to Panda Features in Development

Who is online

Users browsing this forum: Yahoo [Bot] and 1 guest