Usage of Fog

[markjacksonguy]

Is Fog safe to use?

I can set it and even get good results, but there is an error message in the console that says,

"pgraph(error): shader generator does not support fog yet"

So why am I seeing a nice fog effect then? Should I disable fog? If it's going to cause problems then it's best to disable it (although that would be sad since fog can do so much for an environment)

[drwr]

In 1.7.2, the shader generator did not support fog, so you could either (a) use fog or (b) use the auto-shader, but not both on the same node. If you're seeing fog, it must be on nodes that don't have the auto-shader applied. If you're getting this error message, it must be that at least some of the nodes that you are applying fog to also have the auto-shader applied.

For the upcoming 1.8.0, I believe rdb has added fog support to the shader generator, so this is no longer an issue. You could install the buildbot release to get an interim build to develop with in the meantime.

David

[markjacksonguy]

Will 1.8 of P3D release within 2012? and will any apps made with 1.7 be compatible?

Lastly, will 1.8 support auto DOF?

[drwr]

We don't have a timeline for release, but I think it is likely that 1.8 will be ready before the end of 2011. Like all Panda3D releases so far, it will be largely compatible with previous releases. As I said, you can try out the buildbot release to get a preview of what's coming.

No one is working on auto DOF in Panda at the moment, to my knowledge. That's the sort of thing that can be implemented in the application layer, with effort, but it isn't part of the engine. However, if you feel this feature is important and you'd like to help add it, we'd be happy to accept contributions.

David

[markjacksonguy]

Can multiple Fog nodes be set within a scene? I see no indication in the manual that it can be done.

[drwr]

Yes, of course. Like any other attribute, you can apply fog to render, which applies it globally; or you can apply it to any subnode, which applies it only to the nodes at that point and below. If you apply it locally, you can apply a different fog attribute to different nodes.

David

[preusser]

Hello. I just tried the Windows buildbot version of Panda3d and fog is still not supported by the ShaderGenerator.
I understand that it might still not be included with the latest release, but I thought I'd let you guys know if there was a problem.

[rdb]

For the upcoming 1.8.0, I believe rdb has added fog support to the shader generator, so this is no longer an issue.

Huh? I don't think I did, nor do I remember saying I did/would.

Adding exponential fog support wouldn't be hard, but it'd probably be inefficient compared to a post-processing filter, which would be fairly easy to write. In the simplest case, one could use a simple lerp instruction with the depth as factor.

But I'll see what I can do.

[drwr]

Oh, my apologies. I'm sorry, I don't know why I thought that, then.

David

[rdb]

No worries. I did it anyway, just committed the changes. All three fog modes are now supported with the shader generator:
http://rdb.name/fog.png
All fog modes produce an effect identical to the fixed-function pipeline fog.
You'll need to wait a day or so till the buildbots pick up my changes.

I haven't tested DX9, just OpenGL.

[drwr]

Wow, I only have to imagine that something is done, and boom, it's done!

Thanks!

David

[wezu]

I've been waiting for auto-shader fog since September 2009. And it's not even Christmas! Sweet.

[preusser]

Thank you rdb.
I see that you are familiar with the ShaderGenerator class. Perhaps you could fix another issue with the ShaderGenerator? We are offsetting a texture on a lava node and we are experiencing very noticeable fps drops.
We have experienced this on all the GPUs we tested.
Other than that the ShaderGenerator works like a charm and saves us from writing our own Cg code.
I'm sorry if I should have posted this in a separate thread.

[drwr]

This is a known issue with the shader generator: any change to the render state for a given node forces a regeneration and recompilation of the shader applied to that node, potentially quite expensive. This includes innocuous state changes such as adjusting a texture offset.

This is a difficult problem to solve, because it goes to the fundamental way that states and shaders are implemented. We've been thinking about solutions for Panda3D 2.0, but the solution won't be ready any time soon.

So, in the meantime, the usual workaround is to use a hand-coded shader, rather than the auto-shader, on any node that you have to apply frequent state changes to. If you're not versed in writing shaders, you can start with the shader generated by the auto-shader, which you can inspect by setting the config variable "dump-generated-shaders 1".

David

[preusser]

I'm sorry, but if the state change causes recomputation of the shader, couldn't there be an extra conditional statement for shader recomputation set by a method such as NodePath.setShaderUpdate(bool)?
It's just a pity that we can't fully rely on ShaderGenerator and would need to find shader programmers now for only a small number of effects.

[drwr]

It's a much lower-level problem than that. The shaders are cached on the RenderState objects, and there is a unique RenderState object for each unique combination of state attributes.

When you call nodePath.setTexOffset(), you are forcing the generation of a new RenderState object. This new object does not have a pointer to the previously-generated shader, so keeping the previous shader is not even an option.

There are many things that are good about Panda's RenderState design, but the storing of auto-generated shaders in the RenderState object is not one of them. In order to solve this problem, we will need to find a better place to store these shaders. That is the low-level problem that is difficult to solve in the short term.

But as I said, you don't actually have to know anything about Cg. You can still use the shader generator to make all of your shaders, you just have to save the results of a few of them and apply those shaders by hand.

David

[rdb]

Hmm... David, I'm wondering if this dirty hack could work. We could do something like this in set_state_and_transform:

Code: Select all

  if (_target_shader->auto_shader()) {
    CPT(RenderState) new = _target_rs;

    // Filter out attribs that are irrelevant for shader generation
    new = new->remove_attrib(tex_matrix_slot);

    if (new->_generated_shader == NULL) {
      new->_generated_shader = _shader_generator->synthesize_shader(new);

etc etc. This would be done every frame, though, so I'm not sure of the performance implications of that.

[preusser]

Well in that case I can think of another dirty hack.
How about having a method like NodePath.bakeShaderAuto(), which will dump the shader file to Panda3d's cache folder, disable ShaderGenerator on the Nodepath, then apply the dumped shader to the NodePath via setShader() and set the shader inputs automatically? Depending on how these classes work, maybe you wouldn't even need to save it to disk, but just pass the shader as a string.
It would be a hack, but I think it could work and would be better than waiting before ShaderGenerator is redesigned in the not very near future.

[drwr]

These both sound like fine suggestions.

Actually, rdb's idea is similar to one I was just thinking about, but perhaps a little easier to implement. I was thinking about building a hash of the relevant RenderState attributes, and storing the shader in a separate hashtable that all RenderStates shared, but simply hashing to a different RenderState pointer is easier and arguably more elegant. We could then cache that pointer within the original RenderState to avoid the runtime hit every frame, and I think that should solve the problem nicely. We'll just need to take a bit of care to avoid circular reference counts in this cache.

But preusser's bakeShaderAuto() hack is also a nice suggestion in case we can't find a better solution. It basically automates the workaround I had suggested, making it slightly less clumsy.

Let me think about these some more.

David

[Sothh]

These both sound like fine suggestions.

Actually, rdb's idea is similar to one I was just thinking about, but perhaps a little easier to implement. I was thinking about building a hash of the relevant RenderState attributes, and storing the shader in a separate hashtable that all RenderStates shared, but simply hashing to a different RenderState pointer is easier and arguably more elegant. We could then cache that pointer within the original RenderState to avoid the runtime hit every frame, and I think that should solve the problem nicely. We'll just need to take a bit of care to avoid circular reference counts in this cache.

But preusser's bakeShaderAuto() hack is also a nice suggestion in case we can't find a better solution. It basically automates the workaround I had suggested, making it slightly less clumsy.

Let me think about these some more.

David

Nice to see some progress on these issues.

[preusser]

It basically automates the workaround I had suggested, making it slightly less clumsy.

The manual approach requires the user to set up a scene for dumping the needed shader, dump it, then load it manually. Not very tidy, but not impossible. The main problem I see here is people might still need to know some Cg to be able to set up the correct shader inputs.

As for the fog, now that I think about it a filter based one would be better in the sense that you would be able to have your NodePaths with custom shaders affected by it. This could mimic only the exponental fog I think, but would still be a life saver for people using custom shaders.
rdb, we're asking a lot from you, but it would be great if you would implement this feature as well.

[redpanda]

*being impatient*

[Bradamante]

Isn't that the Quake3 custom map The Doom that came to Dunwich?

We have been talking about post-prod fog since 2009. How is it being used in Panda3d?

No worries. I did it anyway, just committed the changes. All three fog modes are now supported with the shader generator:
http://rdb.name/fog.png
All fog modes produce an effect identical to the fixed-function pipeline fog.
You'll need to wait a day or so till the buildbots pick up my changes.

I haven't tested DX9, just OpenGL.

[rdb]

Yeah, I suppose it is that map.

The fog I implemented is not a post-processing filter, it is applied in the shader on the model itself, in order to 100% match the fixed-function fog effect. You can simply enable fog the usual way, and enable the shader generator on the node.

[preusser]

Can someone get this postprocess fog working? http://code.google.com/p/pandademomaste ... p&can=2&q=
Maybe this could be added as a CommonFilter. That way everyone would be happy.

Did you guys decide on the other issue? I'm really looking forward to using texture projection on a per-pixel lit terrain in my project

[drwr]

Did you guys decide on the other issue?

I'm working on it.

David

[preusser]

Okay, sorry for being pushy.

I just tested the demomaster fog filter with the buildbot release and it works now.
I'll see if I can merge it with CommonFilters.

EDIT: Okay, here's a standalone version of clcheung's postprocess fog. http://www.4shared.com/file/LlhIjglw/demo_fog.html

Could someone with Cg knowledge explain how this works? It seems like the exponent density is calculated from "focus" and "fog density" shader inputs. The original non-postprocess method of Panda3d just requires a single ExponentDensity value.

[clcheung]

Okay, sorry for being pushy.

I just tested the demomaster fog filter with the buildbot release and it works now.
I'll see if I can merge it with CommonFilters.

EDIT: Okay, here's a standalone version of clcheung's postprocess fog. http://www.4shared.com/file/LlhIjglw/demo_fog.html

Could someone with Cg knowledge explain how this works? It seems like the exponent density is calculated from "focus" and "fog density" shader inputs. The original non-postprocess method of Panda3d just requires a single ExponentDensity value.

I forgot why there is such a parameter and I have not much Cg knowledge to explain. May be it was there because I was developing a DOF demo at that time....

You can ignore that parameter and use only fog density.

[preusser]

Maybe I should have checked the cg file even though I didn't know the language. It seems the focus is just multiplied to the final value so you can safely remove it.
There's still something wrong.
I think this illustration could explain it better than words:


In other words, if black is no fog and white is completely covered by fog, then the distant objects are never *completely* obfuscated by fog, unless you use really high values, but then you won't be able to see things few units away.

[preusser]

I would expect setFog/clearFog to work the same way as setLight/setLightOff and setShaderAuto/setShaderOff

Code: Select all

import direct.directbase.DirectStart
from pandac.PandaModules import *

myFog = Fog("Fog Name")
myFog.setColor(0.4,0.4,1)
myFog.setExpDensity(0.03)
render.setFog(myFog)

environ = loader.loadModel('environment')
environ.reparentTo(render)

environ.clearFog()

run()


I thought like lights and shadergenerator, fog would be applied to render's children and you could disable fog on the nodes you want later.