Problems with interrogating wglStateGraphicsGuardian

As part of my learning process of how Panda’s build and interrogate system works, I’m trying to interrogate wglGraphicsStateGuardian but am running into some problems.

in wglGraphicsStateGuardian.h, i added

class wglGraphicsStateGuardian : public GLGraphicsStateGuardian {
PUBLISHED:
  int testfoo();

and the appropriate dummy function wglGraphicsStateGuardian::testfoo in wglGraphicsStateGuardian.cxx

Then I amended makepanda with

if (sys.platform == "win32" and PkgSkip("GL")==0 and not RUNTIME):
  OPTS=['DIR:panda/src/wgldisplay', 'DIR:panda/src/glstuff', 'BUILDING:PANDAGL',  'NVIDIACG', 'CGGL']
  TargetAdd('wgldisplay_composite.obj', opts=OPTS, input='wgldisplay_composite.cxx')
  
  # New code
  IGATEFILES=GetDirectoryContents('panda/src/wgldisplay', ["*.h", "*_composite.cxx"])
  TargetAdd('libwgl.in', opts=OPTS, input=IGATEFILES)
  TargetAdd('libwgl.in', opts=['IMOD:pandagl', 'ILIB:libwgl', 'SRCDIR:panda/src/wgldisplay'])
  TargetAdd('libwgl_igate.obj', input='libwgl.in', opts=["DEPENDENCYONLY"])

  OPTS=['DIR:panda/metalibs/pandagl', 'BUILDING:PANDAGL',  'NVIDIACG', 'CGGL']
  TargetAdd('libwgl_module.obj', input='libwgl.in')
  TargetAdd('libwgl_module.obj', opts=OPTS)
  TargetAdd('libwgl_module.obj', opts=['IMOD:pandagl', 'ILIB:libwgl'])  

  TargetAdd('libpandagl.dll', input='libwgl_module.obj' )
  TargetAdd('libpandagl.dll', input='libwgl_igate.obj')    
  # End new code

  TargetAdd('pandagl_pandagl.obj', opts=OPTS, input='pandagl.cxx')
  TargetAdd('libpandagl.dll', input='pandagl_pandagl.obj')

But interrogate has some problems processing libwgl.in

[ 46%] Building Interrogate database built/pandac/input/libwgl.in
---->>>>woutd built/pandac/input/libwgl.in
built/bin/interrogate -srcdir panda/src/wgldisplay -Ipanda/src/wgldisplay -Dvola
tile -Dmutable -DCPPPARSER -D__STDC__=1 -D__cplusplus -D__inline -longlong __int
64 -D_X86_ -DWIN32_VC -D_WIN32 -D_MSC_VER=1500 -D"_declspec(param)=" -D_near -D_
far -D__near -D__far -D__stdcall -oc built/tmp/libwgl_igate.cxx -od built/pandac
/input/libwgl.in -fnames -string -refcount -assert -python-native -Sbuilt/includ
e/parser-inc -Ipanda/src/wgldisplay -Ipanda/src/glstuff -Sthirdparty/win-python/
include -Sthirdparty/win-libs-vc9/nvidiacg/include -Sbuilt/tmp -Sbuilt/include -
DMAKEPANDA= -DBUILDING_PANDAGL -module pandagl -library libwgl config_wgldisplay
.h wglGraphicsBuffer.h wglGraphicsPipe.h wglGraphicsStateGuardian.h wglGraphicsW
indow.h wgldisplay_composite.cxx wglext.h
      *** Error in /c/panda_src_1_80/panda3d/built/include/panda_glext.h near li
ne 3453, column 2:
      parse error
Error parsing file: 'wglGraphicsBuffer.h'
Interrogate failed, retrieving debug output...
'built' is not recognized as an internal or external command,
operable program or batch file.

I took at look at panda_glext.h, line 3453, and it seems to be choking on a very innocent looking

typedef __int64 int64_t;

Anybody have any ideas?

The problem is actually the line before, I think:

typedef __int32 int32_t;

The problem is that __int32 is a windows-specific type, it’s not part of the C++ language specification, so interrogate doesn’t know what it is and can’t figure out the typedef. The same would be true of the __int64 line, except that we explicitly add the flag “-longlong __int64” to the interrogate command line, which tells it to understand __int64 as the “long long” type. (Don’t know why Microsoft decided it needed its own nonstandard type names, but that’s Microsoft for you.)

The bigger issue, though, is that glext.h was never meant to be parsed by interrogate, and might have many other problems as well, with this or other similar Windows-specific code. We usually solve this problem by bracketing the relevant code with #ifndef CPPPARSER … #endif (you could protect the include panda_glext.h line this way, for instance), or we shadow the whole file by creating a dummy file in parser-inc, which interrogate will read instead of the actual file. I’m not sure if the parser-inc trick will work for a file within Panda’s own source tree, though.

David

If I understand correctly, then other non standard variables in
wglGraphicsStateGuardian.h should also be protected using
#ifndef CPPPARSER … #endif ?

For example: wglGraphicsStateGuardian.h

 ...
 bool get_properties_advanced(FrameBufferProperties &properties, HDC hdc, int pfnum);
  void choose_pixel_format(const FrameBufferProperties &properties, bool need_pbuffer);
  virtual void reset();

#ifndef CPPPARSER
  INLINE HDC get_twindow_dc();
#endif

  static bool get_gamma_table(void);
  static bool static_set_gamma(bool restore, PN_stdfloat gamma);
  ...

Yes, though that quickly gets ugly. An easier way is to add a bogus typedef for HDC to dtool/src/parser-inc/windows.h, along with all of the other Windows typedefs there. For that matter, you could add an __int32 typedef.

David

I managed to get everything to compile by using a lot of #ifndef CPPPARSER. And then at the end I discovered that base.win.getGsg() was only giving me a base gsg not a wglGsg. So I can’t test my wglGsg extension without doing a lot more hacking. But I will assume that because everything compiled, what I did was correct

Thanks David!

There two things that I’m still confused about,

1. I don’t see an explicit definition to dtool/src/parser-inc/windows.h in Makepanda.py; A) Is interrogate always looking in this directory B) How does it know to look in the shadowed windows.h for things like HDC? Is it because wglGsg included <windows.h>? Or all of the shadowed .h files mashed to together to form a giant ghost list? C) In case of name clashes between several external libraries, how does interrogate discriminate between what to use? I guess in some sense it doesn’t matter if you’re just ghosting things, but then why not toss all of these typedefs into one giant file? Just purely organizational/logistical reasons?

2)To make the extension, I have to make an intermediate libwgl.in using

TargetAdd('libwgl.in', opts=OPTS, input=IGATEFILES)
  TargetAdd('libwgl.in', opts=['IMOD:pandagl', 'ILIB:libwgl', 'SRCDIR:panda/src/wgldisplay']) 

I’m not quite sure what IMOD:NAME and ILIB:NAME stands for exactly. I’m guess that IMOD:NAME always refers to the super dll libNAME.dll and ILIB:OTHER is referring to a temporary used to accumuluate submodules?

So several source files->ILIB
several ILIBS form a IMOD?

Great! I do prefer to limit the #ifdef’s in the code, for readability, so if you find a way to replace some of these with equivalent shadowing in parser-inc, that might be good too.

Hmm, that means that something’s going wrong with the automatic downcasting. The way this is supposed to work is this: when interrogate wraps a function that returns a pointer to a class that inherits (eventually) from TypedObject, then it also adds a bit of code to call ptr->get_type() before it returns, and if the TypeHandle returned by get_type() indicates a classname that it knows about, it will convert the pointer to the specified Python type before returning it.

So, if getGsg() is not returning a wglGraphicsStateGuardian, one of the following must be going wrong:
(1) gsg->get_type() (or, in Python, gsg->getType()) is not returning an appropriate TypeHandle that indicates
“wglGraphicsStateGuardian”, or
(2) Python doesn’t have a class definition for wglGraphicsStateGuardian for some reason, or
(3) Python doesn’t know the inheritance chain from GraphicsStateGuardian to wglGraphicsStateGuardian.

Actually, glancing at the code, I see that the problem is almost certainly (3). You’ll need to publish GLGraphicsStateGuardian as well, because that’s the intervening class. It needs to at least expose one method of that class for Python to see it.

We add parser-inc to the include file search path on the interrogate command line, so that when interrogate reads #include <windows.h>, it finds the file in parser-inc before it finds the one in the system.

If there is a name clash between different external libraries, then we have bigger problems, because the actual C++ code is going to experience the same name clash. Fortunately such name clashes are rare, but when they do occur, we have to take pains to isolate the parts of the code that use each conflicting name, so that no part of the code includes both clashing header files. This also solves the same problem with interrogate.

The parser-inc files serve two purposes. One, it defines the required typedefs or class definitions. Two, it shadows the actual header file, which interrogate is going to choke on anyway. If we had only one giant file, we would meet the first purpose, but not the second; so we need to have the individual files anyway. And therefore it makes sense to organize the typedefs into the appropriate files, instead of lumping them all (confusingly) together. Also, not all the C++ code includes all of the external library files at the same time.

You’ve got the right basic idea, though the ILIB name, by convention, should be the name of the directory, with lib prepended. So it should be libwgldisplay, not just libwgl.

The reason for IMOD and ILIB has to do with the way Panda is built on Linux and OSX, and how it is built (differently) on Windows. On the Unix-based platforms, a different .so file (similar to a .dll) is compiled for each directory, which is a good design that gives the most modular code. Each directory produces one “library”, which is named after the directory itself, with “lib” prepended, according to the Unix convention.

On Windows, this design doesn’t work, for various fiddly reasons having to do with poor design on Microsoft’s part for the dll format, so we instead have to compile all of the individual libraries together into a small handful of dll’s. These dll’s are called “metalibs” or sometimes “modules”. On Unix, the modules exist, but they’re just stub files that actually reference the individual library files. On Windows, the modules are the actual code, and the library files don’t exist. But interrogate (and makepanda) has to know about the modules and the libraries, and that’s the reason for the two parameters.

David

When I try to publish GLGraphicsStateGuardian, by interrogating glGraphicsStateGuardian_src.cxx, interrogate seems to have problems with the macro CLP such as

int CLP(GraphicsStateGuardian)::testfoo3() {
  return 5;
}

I can sort of ‘cast’ away the problem by explicitly defining what CLP is for the parser such as

[code]

#ifndef CPPPARSER
int CLP(GraphicsStateGuardian)::testfoo3() {
return 5;
}
#else
int GLGraphicsStateGuardian::testfoo3() {
return 5;
}
#endif

But this seems like an ugly hack. Is there a way for interrogate to understand the CLP macro?

You’re not supposed to interrogate that file directly. You could try interrogating the file that includes it, such as glgsg.cxx, but these files are generally not meant to be interrogated at all.

Yes, I remember what you told me. Hence, I’m curious to what David thinks would be best way to expose GlGraphicsStateGuardian.

You’ll need to add interrogate to glgsg.h/.cxx. These are the files that actually compile glGraphicsStateGuardian_src.cxx and related files.

As rdb points out, these files were never meant to have interrogate run on them, so there might be issues; but I don’t think the issues will be worse that what you’ve already encountered with wglGraphicsStateGuardian.

David

Ok. But should interrogate understand the CLP macro? It’s currently stumbling at what appears to be CLP for me. I could #ifndef them away one by one, but if interrogate should work understand CLP, then I have setup something incorrectly in TargetAdd in makepanda.

Yes, because it is defined by glgsg.h, which defines all of the relevant macros (including CLP), and then includes glGraphicsStateGuardian_src.h. This is the same way that the linmath directory works, which works fine with interrogate, so I see no reason why it shouldn’t also work with interrogate here.

The point is that you can’t interrogate glGraphicsStateGuradian_src.h, or anything in glstuff, directly; those files aren’t meant to be read in isolation. You can only interrogate glgsg.h and the other files in the glgsg directory.

David