I’m trying to switch to the 1.10 development version but I run into some issues. I don’t know if this is the place to report these, but I guess someone will tell me if it isn’t.
First to make sure I do the installation properly.
I’m running on windows 10 64 bit, so I’ve downloaded the latest available .exe with x64, which is at the time of writing 3 builds ago. After installation, I switch my projects python interpreter to the python.exe in the new 1.10 folder. I’m using python version 2.7.
I run into issues with textures blended with alpha channels, particularly when the alpha is loaded from a different texture. I’m using the TextureStage.setCombineRgb(…) and TextureStage.setCombineAlpha(…)
In 1.10, only setting a setCombineAlpha (so no setCombineRgb()) to a texture stage crashes Panda with the error:
AssertionError: shader != nullptr at line 1576 of c:\buildslave\sdk-windows-amd64\build\panda\src\pgraphnodes\shaderGenerator.cxx
In 1.9.4, I have no problems with this.
If I make sure at least .setCombineRgb(…) is available and maybe accomplished by a .setCombineAlpha(…) there are no crashes, but blending does not work. I seem to get no blending results from the texture, only one texture is visible. Another problem is that the result seems to be black and white (no colors). I use the separate texture stage for the alpha channel to set it to a different UV set.
Also, I project a reflection buffer texture on water, my terrain overlaps with this where there is land. This texture is also projected on the terrain in 1.10, but maybe this is caused because there is some transparency on the terrain.
I’m really looking forward to use 1.10, any help would be great!
Could you post the precise settings that aren’t working? A fair amount of work was done in 1.10 to ensure that blending works more correctly in the shader generator than in 1.9, but it’s possible that some cases were missed.
Thanks for the supersonic response. I have made a test program that reproduces it, I’ve attached it to the post. If you need more info, let me know.
Running this example in 1.9.4., I get both diffuse textures blended properly. In 1.10, I only see the first texture, as if the alpha texture is completely black.
Yes, there was a regression with alpha blending. I’ve just pushed a fix. New buildbot builds are available now.
Let me know if you encounter other issues with texture stages.
Great that works on that issue. I still have some other issues but I certainly hope the performance is what I get with these solved.
Today I could only do a quick test, unfortunately I wont be able to do a more detailed test before Friday. One obvious issue is that there are no shadows or specular. I blend these with TextureStage.CSPrimaryColor. Also some aircraft seem to randomly flicker to a very bright diffuse and the wings are black. As I use various texture layers to apply reflections etc, I will have to test in more detail what works what doesn’t. The reflection layer is not visible, some other blended textures blend properly. The last Obvious thing is that objects which have an alpha texture and a transparency attrib on it, do not show transparent.
If that helps I could do a more detailed test on Friday and see if I can update the test program.
There have been numerous changes in 1.10 to the shader generator in an attempt to make the texture blending behaviour better match the fixed-function pipeline. It could be that there is a bug, or that your application relies on the incorrect behaviour from 1.9. Did your application work correctly in the fixed-function pipeline in 1.9?
I’m happy to help you get to the bottom of the issues; I can help you debug if you privately gave me access to your application. Otherwise, it would be helpful to get test cases if you narrow down the individual issues. (If you are having trouble figuring it out, it’s also an option to generate an apitrace file so that I can step through the OpenGL calls and find out where it’s wrong, but it’s more helpful to first isolate the problematic areas.)
As for alpha blending, the only thing I can think of at the moment is that the default alpha blending operator has subtly changed in 1.10; you can try setting “old-alpha-blend true” in Config.prc to enable the old behaviour and eliminate this as a potential factor.
What do you mean when you refer to the performance? Have you seen a performance increase or decrease after upgrading to 1.10?
Yes the performance is way better (+50%!) but not everything is working yet. After trying the find the problematic areas I did solve a lot of issues but I still have some:
It seems that the texture stage with the normal map may not be the bottom layer (lowest sort order). If I do that, some objects appear black and some objects show the normal map as a diffuse. Now I made sure my normal map layers are the top layers (highest sort order) almost all blending problems are fixed.
A remaining issue is that I can’t seem to get any lighting effects on objects in my airport scene, my other scene, the globe view scene has less problems with that. If I try to reproduce the issue in a small test program, I can’t reproduce it. It probably has something to do with my set up of lights or something.
There is an issue with the shadow caster together with my set up as well, possibly related to the lighting problem. My game starts in the globe scene which uses no light with a shadow caster, in the airport scene a light is set with the panda shadow caster. Although I don’t see lighting effects (issue above) nothing obvious happens. When I switch back to my globe view 1.10 freezes. If I comment spotlight.setShadowCaster(shadows_on, shadow_res, shadow_res), this freeze doesn’t occur. Again, if I make a small test program which toggles a light with shadow caster with scene.setLight(spot) and scene.clearLight(spot) like in my set up, all works as expected in 1.9 and 1.10.
Additionally, not a single node has specular in my airport scene, in my globe scene the specular seems to be working normal.
“old-alpha-blend true” has no effect at the moment, commenting render.setShaderAuto() in 1.9 makes a couple of blended textures are black or white and the lights don’t work as well.
For now I’ll play around some more to find out what settings/functions affect the issues. Any suggestions about where to look are helpfull but I guess you’d have to know a bit more about my set up. I will let you know if I find something.
I have been doing some testing and adressed the most important issues:
I have set some lights to nodes lower in the scene graph to control which objects are lit by which lights. This causes the light on these nodes to appear disabled (no shadows/specular…) in 1.10. In 1.9 the lighting appears as expected. I have attached a test program that reproduces it. Maybe I have to use bitmasks or something to exclude objects from certain lights, let me know.
Some other issues had to do with what appears the default way of processing alpha by a texture stage. I didn’t get to the bottom of this but it seems that when a texture stage has a setCombineRgb() call and no setCombineAlpha() the alpha is processed as set to black (0.0) in 1.10 instead of white (1.0) in 1.9. A difference is visible if you use a RGB texture with one texture stage, use setCombineRgb() with replace and set no setCombineAlpha. If you then apply a transparency attrib the node is invisible in 1.10 and visible in 1.9. As a final set up this makes no sense of course, but if it affects how multiple layers of texture stages are processed the result is different. Maybe this has to do with matching the fixed-function pipeline. It might be that I relied on implicit behaviour here and I just need to add some setCombineAlpha() calls.
I still encounter the freeze that seems to be related to the shadow caster, but maybe that’s gone when my lights works properly.
Other then the issues described above there are lot of advantages as well with using 1.10. The fps seems to settle at about the same as 1.9, I can’t really say yet because not everything works. It’s higher now but I have disabled some lights. The fps consistency however is way(!) better. In 1.9 I have"reloads" when I move around the scene so I have a lot of stutters and in 1.10 if I have none so that is great! Also, I saw in the sample programs the shader terrain, that looks really easy to implement and very effective for what I want. Does it use tesselation, I couldn’t find a tesselation shader? Not that it should but I’m just curious how it works.
In the sample program I have included one more question.
Thanks for the clear test case! I’ve just checked in a fix for the issue with lights not being applied properly.
Great to hear! The shader generator got a significant overhaul precisely to address the problem of stuttering while moving around a scene. The regressions are a by-product of this overhaul; there are a lot of corner cases in the shader generator that can be overlooked. Glad to have you testing it so we can track these down early.
It doesn’t use tessellation shaders; it uses fixed geometry that is generated on the CPU that is deformed using a vertex shader on the GPU.
I happened to run into this issue by accident; it only happens when lighting is disabled while a normal map is applied. I’ve checked in a fix.
As for the other issues, I’m afraid I can’t reproduce any sort of freeze. Are there any settings that have been enabled? If you are using the multithreaded pipeline, there was a bug with the shadow implementation that caused a freeze, but it has already been fixed.
I also can’t spot an alpha-related issue in your sample. Perhaps this is another thing that has recently been fixed?
This is what I get when I run your code with Panda 1.9 and 1.10:
The difference may be caused by the order in which texturing and lighting are applied in the shader generator. In 1.9, the shader generator applied lighting information after the texturing calculations, which meant that you could not eg. use an emission texture to add additional lighting values. This was not consistent with how the fixed-function pipeline worked, so in 1.10, the lighting calculations are done first and stored in CSPrimaryColor.
If you’d like to continue using the old behaviour, let me know and I can add a configuration variable to restore the old behaviour.
Well, that creates a texture with a width of 0, which isn’t supposed to be allowed. That said, I noticed the OpenGL renderer handles it just fine (creates a 1x1 texture) so I just checked in some changes to make sure the other renderers are as lenient with it, too.
mmm… on my PC, If I don’t modify anything, I don’t see a difference between the two versions, both versions give the same result as your 1.10. I see the difference if I place the second spotlight such that light actually becomes visible on the plane, but then the 1.10 way gives a more life like result. Additionally, I used to have some issue with reflections on my aircraft with dimmed night light in 1.9, but this seems to be automatically solved as well, it possibly has something to do with this. To speak for myself, I prefer to proceed with the 1.10 way
Well the blending are completely solved as far as I can tell now. The issue with the shadow caster/ freeze remains. If I disable the shadow caster I don’t get a freeze. I use threading-model /Draw, if I disable this the freeze does not occur either (with the shadow caster on). However I have not been able to create a test program that reproduces it so it must be related to other settings in my project. Do you have a proposal how we can proceed with this? I would not mind giving you access to my project however I must warn you I’m not a very experienced programmer. I don’t know if an apitrace could be helpfull? I would have to figure out how that works. Please let me know what would be an effective way and if you want to help with this issue.
I have also been messing with the shader terrain and I’m very glad with the result! I’m currently trying to learn and gain experience in GLSL shaders and got my scene lit with normal maps and blended textures. I have one issue with shader terrain. When threading-model /Draw is set the chunks flicker in the corner when moving the camera. The chunks that flicker are always in the corner/on the side of my screen, in the positive x and y direction in relation to my scene. I can see the issue in the shader terrain sample program as well.
One more thing is a crash once on a random moment when my game was running for a while.
Traceback (most recent call last):
File "C:\Panda3D-1.10.0-x64\direct\showbase\ShowBase.py", line 1908, in __garbageCollectStates
RenderState.garbageCollect()
AssertionError: !this->was_deleted() at line 192 of c:\buildslave\sdk-windows-amd64\build\built\include\weakPointerTo.I
Traceback (most recent call last):
File "C:/Panda3D-1.9.4-x64/Games/main.py", line 444, in <module>
engine.run()
File "C:\Panda3D-1.10.0-x64\direct\showbase\ShowBase.py", line 3040, in run
self.taskMgr.run()
File "C:\Panda3D-1.10.0-x64\direct\task\Task.py", line 531, in run
self.step()
File "C:\Panda3D-1.10.0-x64\direct\task\Task.py", line 485, in step
self.mgr.poll()
File "C:\Panda3D-1.10.0-x64\direct\showbase\ShowBase.py", line 1908, in __garbageCollectStates
RenderState.garbageCollect()
AssertionError: !this->was_deleted() at line 192 of c:\buildslave\sdk-windows-amd64\build\built\include\weakPointerTo.I
It only happened once so I do not have much more info then this.
Last thing for today is a general question about texture blending with GLSL. For my aircraft I blend various textures for livery colors, aircraft details, reflections and night maps. Some use additional alpha textures to mix the layers properly. The thing is that 4 separate texture files could be mixed into 1 static texture (with pnmimage for example) because the blending of these layers do not change at runtime and the uv coordinates they use are also the same. I don’t want to store the combined texture files because I would get an excessive amount of textures (Im guessing dozens of GBs). Now the fragment shader runs for every fragment, every frame right? So doing the blending of this 4 textures here would keep your PC busy with blending them while that actually only has to be done once. Would it be wise to blend them using pnm, which is obviously quite slow, or is there another way to only blend them once?
Hmm, I do know there was a bug with a shadow freeze in the multithreaded pipeline, but this had been fixed. Can you run the shadows sample program with your threading-model setting? What would help me the most here is a stack trace, but on Windows this is not trivial to obtain, so being able to run your code on my computer would be quite useful too.
There are still some known bugs with the multi-threaded pipeline. The assertion error you are getting seems to be one of them. I’m not sure I can do much about that specific case without a stack trace from a debugger, though. I’ll keep an eye open if this happens again.
The shader terrain appears not to be compatible with the multithreaded pipeline. I’ve noted this for a future task.
If it is really dozens of GBs, then it won’t feasibly fit in GPU memory, and you also would pay a performance cost when uploading the textures to the GPU. In that case you don’t have much choice but to do the blending in the shader.
Yes, I may toggle the light with no problems like in my own test program.
Ok I’ll PM you probably later this week.
I have only encountered this once and I haven’t encountered it in quite some running time so it doesn’t seem that much of an issue.
Cool!
I think we have a slight misunstanding but it implies an answer to my question. So just to be sure, My situation is that I will never need all the textures at the same time in the GPU, only the textures for the liveries and different aircraft types present in the scene (upto a 100 is the target). However, the different unique aircraft types and liveries that could potentially become present in the scene, are a great amount, at this time over a 100 aircraft types and over a 100 liveries. So precombining all the textures for aircraft types/liveries with for example pnm and storing those combined textures to disc, would create a lotttt of textures (>10.000).
That being said I’m considering two options:
Combining the textures for 1 aircraft to 1 texture with pnm on runtime, before the aircraft becomes present. On my reasonably fast system this takes just under 0,1s (which causes a leg obviously).
Apply all the textures to the model and blend them in the fragment shader. Having to blend 3 MB of textures to 1 MB of textures there, I assume this 3 MB is needed in the GPU memory.
You answer however, implies to me that if I blend textures in a shader, the blended result is the only texture uploaded to the GPU memory. Is this true?
No, that’s not the case. All textures you are blending would be uploaded to the GPU and you would pay the cost of accessing all those textures for every fragment you are rendering.
Unless, that is, you use render-to-texture or a compute shader to do the blending once and store the blended result in a new texture, and unload the original textures. You would still pay the cost of uploading the textures, though.
Blending with PNM isn’t a bad idea then, if you’re willing to pay the CPU cost (ie. increased load time for your game). You could probably cache it on disk after you’ve blended a particular combination. That said, if your current approach is working, I suggest you don’t spend too much time on it unless you’ve identified it as having a significant impact on your performance.
