This is just playing around for now; no idea whether this will work or not; there’s a big risk of critical blocking issues.
Started to look at the possibility of using swig instead of interrogate to build the Python interfaces for Panda.
Swig is a de facto standard for Python / C/C++ interface code generation. It was first developed in 1995, and has grown from there.
Potential advantages of using swig instead of interrogate are:
- perhaps it runs faster?
- swig is fairly standard, so knowing how to use it is transferable knowledge; both to and from Panda development
- possibility for Panda to benefit from future enhancements to swig; and of course Panda can contribute its own enhancements, which benefits everyone
Potential critical blocking issues include, but are not limited to:
- Swig doesnt really understand macros too well
- Swig doesnt like template classes
- Swig cant handle nested classes
- Swig ignores namespace{} s
- other stuff that I havent found out about yet
Introduction to using Swig
Swig is essentially similar to interrogate, but it uses an interface file (.i or .swg) to define its interface. Interrogate reads the PUBLISHED: sections of the headerfiles.
.i files are not really complicated: they’re essentially just the original headerfile with all the #s removed, macros removed, and only the member functions you actually want published present.
Generating an interface file for Swig
If we wanted to use Swig, one approach would be to simply convert the whole project to use .i files rather than the PUBLISHED: sections. Clearly this is just not going to happen, and besides, the PUBLISHED: syntax is rather good.
So, what we need to do instead is to take the current headerfiles and use a code generator to generate the swig file(s) directly from them.
This is not really hard in principle. There are a few issues in practice:
- Swig doesnt like macros, so we need to filter those out, or expand them
- We may have to generate swig declarations for each type T used for each template class, ie %template PointerTo< myclass >
- Swig only makes one pass, and expects its interface files in dependency-sorted order
The following code addresses most of these isssues. There are two parts:
- initial pass through all headerfiles, generating swig-compatible code in rawpanda.swg
- multiple passes through the contents of rawpanda.swg to generate the dependency sorted file pandasorted.swg
Using the script below
Note that the output of this hasnt been fully tested with swig yet; so there are probably outstanding issues. This is a work in progress, definitely not finished yet, not even at feasibility study level.
You will need to run this from the same directory you would run makepanda.bat from, ie the parent of panda, makepanda, dtool etc. Before running it, you will need to modify the paths at the head of the file: rawswigfilepath, unsortedswigfilepath, sortedswigfilepath.
Code description: initial pass
The initial pass walks through all the headerfiles in the panda/src directory.
- It strips comments, and everything starting with #
- it strips all members not in a PUBLISHED: section
- puts public: instead of PUBLISHED: at the start of the class
- replaces PT( class ) with PointerTo< class >
- strips EXPCL_xxx macros
- strips nested classes
- replaces INLINE with inline
- can be adapted to strip / process other macros (eg CPT( class ) -> CPT< class >)
It produces a single file rawpanda.swg, that contains all the class interfaces for the entire source tree, but not in dependency-sorted order.
Code description: dependency sorting
This takes the output from the intiial pass, ie rawpanda.swg, and creates a new file pandasorted.swg, in dependency sorted order.
To save memory, we do this in batch-mode, using multiple passes.
We create a list to store the names of sorted classes, then start reading through our input file.
We read class by class, and check its dependencies.
If the class has no dependencies, or all dependencies are in the list classes we already sorted, we write the class out to pandasorted.swg, and add the classname to our list of sorted classes.
Otherwise, we write it to pandaunsorted.swg.tmp, and continue.
When we get to the end of our input file, we move pandaunsorted.swg.tmp to pandaunsorted.swg, and use that as our new input file. Rinse and repeat.
We make 10 passes, which is sufficient to sort everything that is going to be sorted.
At the end we should probably just tag whats left in unsortedpanda.swg onto sortedpanda.swg, but I didnt add that in yet.
This runs very quickly, since the only thing in memory at any one time is:
- the text for a single class
- a list of classnames
Timings
No timings avialable for swig yet.
Timings for initial pass: about 5 seconds; mostly this is the time spent writing the filepaths to the console
Timing for dependency sorting: < 1 second
Hugh
(Script moved to: see links in later post below)
It doesnt seem really widely advertised. It does pretty much everything the full Visual Studio IDE one does.
). But that does cut both ways, too, because that means someone needs to maintain the interface to swig, as both swig and Panda continue to grow.