COLLADA importer

[rdb]

Howdy guys, it's me again.

Because no one offered to write a collada importer (right here), and I really like the collada format, I was so stupid to say I'd write one, and now I guess I'm stuck to it for the next days/weeks/months.
Anyways, I'm making it in C++, and just like all the other somethingToEggConverters.
I'm doing this in my spare spare time, so don't expect this to be done in the next release already
If somebody offers to help coding, that would be greatly appreciated.

EDIT: Oh, sigh. I first started writing it in collada DOM, until I noticed that it came with another 4 thirdparty libraries, libxml2, boost, pcre and tinyxml. FCollada depends on its turn on cg(no prob) but also on DevIL(ahhgr).
Does panda have some builtin xml parser? Otherwise I think I'm going to grab something like libxml or tinyxml, cuz I think the last thing Panda3D needs is another 5 thirdparty libs.

[Hypnos]

One thing you should consider if choosing a lib...
Look for platform independency. If we have lib in the requirements which is unlikely to be ported to osx/linux/windows in the next time it would be sad.

If it's impossible to find one, adding the support for it is better then nothing

[drwr]

Panda has no xml parser of its own. Feel free to grab the XML parser of your choice. As Hypnos suggests, portability is one of the key important factors.

David

[rdb]

Ah, thanks. I'm just going to try out tinyxml (means no lib at all, just a few source files to add), if that wont work out i'll choose a bigger lib. I'm kinda tired of this thirdparty compile hell on windows.
By the way, does panda have c++ classes which will make my life easier, such as string parsing functions, str2int conversion (and back), etc, etc. Otherwise I would have to implement those myself, which is no problem but I try to avoid duplicate work.

[drwr]

There's a handful of handy functions in panda/src/putil/string_utils.h. In particular, it has string_to_int(). For going int -> string, we normally use the ostream operator; if we want to do this in-memory, we use ostringstream.

David

[treeform]

why not use python for dea2egg ?

see my obj2egg
http://panda3d.net/phpbb2/viewtopic.php?t=3378

I don't see why tools have to be in c++ in order to be included or even fast. I think python is great for this kind of work.

[drwr]

One advantage to using C++ instead of Python for the model converter is that it can then be integrated into Panda's existing SomethingToEgg framework, which allows the low-level importer to load your dea files directly, e.g. loader.loadModel('foo.dea'); and you automatically get the features built into Panda's common model conversion process, like optional scaling or normal generation.

It also means that non-Python users will have an easier time converting files, since they wouldn't have to deal with any of the Python issues that might come up otherwise.

Of course, there are other advantages to coding in Python instead of C++. I'm not in a position to dictate one approach or the other. Since pro-rsoft is volunteering to do this gratis, I'm happy that it gets done either way.

David

[Crimity]

You rock pro-rsoft
Another project of mine is uses collada, so this will make it a lot easier to move back and forth between the two.

[rdb]

Ah. Thanks for all your replies, guys.
drwr, those were exactly the functions I needed, thanks alot.
treeform: I'm going to do it in C++, for the reasons that drwr said, plus, I want to gain more experience in C++ programming.

I just have one question, if I am going to use TinyXML, it only consists of, like 5, files. It's manual page says I could either just put the source files in some panda dir and let makepanda compile it, or I could also compile it first as library and use it as thirdparty package. What would you do?
If you say just include the stuff in the panda source, in what directory would I put it then? Just pandatool/src/dae/ or so?

[drwr]

If there's no reason to modify TinyXML, I'd recommend putting it in the thirdparty directory. This keeps the two projects better isolated, and makes it easier to upgrade to a new version of TinyXML if one comes along.

David

[rdb]

Ah. Thanks.
Would this mean we just put the five tinyxml source files in the thirdparty directory and have makepanda compile it into a lib (it just takes about 22 seconds to compile here), or put a pre-compiled library there?

PS. I've been working on the exporter fulltime for a few days and its really getting along well.

[drwr]

Generally, the thirdparty directory consists of precompiled libraries. So go ahead and compile it and put a library there.

David

[drwr]

On second thought, maybe it's easier just to mix the source in with your own, and not mess with thirdparty. I dunno. Your call.

David

[rdb]

Ah, ok. It's licensed under zlib so I would basically be able to mix it with anything I want, even modify it. I'll just see what will work best later on.

Little question: Should cameras and lights be imported as well?

[drwr]

We don't yet have an interface in egg for representing the properties of cameras and lights. You can import their node positions and orientations, though. This is what we do with maya2egg.

David

[rdb]

Okay, I thought I'd give you guys a status update.
I chose FCollada anyways. I went for tinyxml first but then found myself duplicating a lib like fcollada and buried in a bad C++ hell. FCollada works much better for me.

Here's what supported:
* Integrated in SomethingToEggConverter, useable from ptloader so I can directly pview or load .dae files.
* Scene graph with transforms.
* Basic geometry, including normals, uvcoords, binormals, tangents, etc.
* Materials. (I think even multiple.)
* Multitexturing.
* Normal maps, gloss maps, and a few more of those.
* Basic skeleton hierarchy.
* NURBS curves (untested though.)
* Multiple UV sets.
* Color blend attribs and transparency.
* Some small stuff that isn't worth mentioning.

Here's what is basically there but untested/doesn't entirely work yet:
* Skeletal animations don't work as I want them to.
* Animated morph targets.
* It makes no distinction between phong, lambert, etc yet.

Here's what unsupported yet:
* non-nurbs curves.
* Cg shaders.
* Everything not listed above. Tell me if you have a feature request.

Cute little collada duckie having textures and materials, exported through my thing.

PS. If you have some collada .dae files lying around for me so I can test the converter, that would be great.

[rdb]

Because some people requested it, here's a windows build of dae2egg:
http://bin.pro-rsoft.com/dae2egg.zip

[drwr]

This is really exciting! This is going to open up a huge world of tools for Panda.

David

[treeform]

Found errors!

The number #3 feature i need hierarchy is broken.

op on importing:
http://aff2aw.com/affdata/panda/ranger2.dae

i get

Code: Select all

Reading ranger2.dae
Warning: unsupported instance type found

opon reading the generated egg i get

Code: Select all

Loading ranger2.egg

Warning in ranger2.egg at line 8, column 30:
  <Scalar> texture_type { 2d }
                             ^
Unsupported texture scalar: texture_type


Warning in ranger2.egg at line 15, column 30:
  <Scalar> texture_type { 2d }
                             ^
Unsupported texture scalar: texture_type


Warning in ranger2.egg at line 22, column 30:
  <Scalar> texture_type { 2d }

And when the entire thing loads the turrets that supposed to be in different parts of the ships are located at 0,0,0 with wrong scale and rotation. I think you are not multiplying the matrices right.

Thanks pro for all the hard work!

[rdb]

The warning just says that it found something like a camera node or a light node which the egg format doesn't support yet, so I'm not converting that.

The EggTexture error is related to a bug in panda. Is fixed in cvs and will be fixed in 1.5.4, for now, edit the egg file and replace "texture_type" with "type" everywhere.

Upon a quick look, your .dae file uses a very funny way to describe transforms. It seems to have both <translate> and <matrix> elements, so it could be that my converter is first parsing the matrix, but then overriding it with the translate, which is 0, 0, 0.

Thanks for reporting this bug, I'll take a look at it. I'll update the .zip file above when I manage to fix it.

[treeform]

Thanks alot! How is bones support? My team uses bones to layout potions of stuff for ut3 and wonder if we might use that setup for 2aw. Well test bones and tell you if something breaks.

[rdb]

Joints should basically export fine, although the animations are not working well yet.

Well, I've tried out your model and it looks fine to me - can you post some screens showing how it is supposed to look?
(Also, the textures are missing.)

[treeform]

Did you change any thing with the importer?

All the turrets are located inside the model at 0,0,0
you have to turn on wire frame to see them. They should be smaller and located on the little turrets platforms on the sides, top and bottom not at 0,0,0 in side the model.

[rdb]

I see the turrets. I still don't understand -- getPos(render) on the turrets definitely doesn't return 0, 0, 0 at all.
Do you have a reference .egg file that actually is exported fine? So I can compare the files.

[rdb]

I've managed to fix the bug. Was a really stupid bug: I just had to replace all <Group> nodes by <Instance> nodes. The turrets show up fine now. (Or is it too ugly to do it this way?)
I've uploaded it on the same URL.

Another question: collada files support having instances to other nodes. Reading the EggSyntax file, it says this regarding instances to other nodes: (using the <Instance>{<Ref>{}} stuff)

Code: Select all

A special syntax of <Instance> entries does actually create shared geometry in the scene graph.  However, please note that this special syntax is not supported by Panda3D at this time.  It is documented here against the day that it will be supported.

Has this changed? I ask this since this document is very old, so it might be outdated. If it's implemented, I can directly translate it from collada. If not, would I then just duplicate the referenced node myself if multiple instances occur in the file? (The latter is what I'm doing atm.)

[drwr]

Or is it too ugly to do it this way?

That's acceptable, but because you have to do this, it means you have violated one of egg's principle tenets: the vertices should be stored in global coordinates, not local to their transforms. It's a weird convention, because virtually every other 3-d model format stores vertices in local coordinates, but it does have some nice advantages (for instance, you can easily hand-edit an egg file to adjust the transforms, or take them out altogether, without disturbing the vertices).

Fortunately, the egg library makes it easy to follow the egg convention, and apply the appropriate conversions to your vertices, from local to global, as you add them to the vertex pool. Just multiply them by egg_node->get_node_to_vertex().

Has this changed? I ask this since this document is very old, so it might be outdated.

It hadn't changed before yesterday, but when I saw your post, I thought, gee, I've been meaning to make that change for a while, and I don't think it's very hard. So I did. So, as of today, you can use the <Ref> syntax as described in the document to store true shared instances.

David

[rdb]

Awesome! You rock man! Thanks alot!

EDIT: hmm, this is going to be a little bit tricky. I have my function like this:

Code: Select all

PT(EggGroup) DAEToEggConverter::process_node(const FCDSceneNode* node) {

Inside the function, the EggGroup is created, transforms applied and returned. In the function that calls this one, I do the reparenting. Got any ideas?

[drwr]

The only solution I can see is to rework the function to do the parenting first. The node's parentage is critical information for determining the proper coordinate space of its vertices, so you can't really assign vertices until after the node is in the right place.

Or, you can continue to keep all the groups as <Instance>s. That will work fine, though it's not standard egg. It might also give you trouble when it comes time to making animations work properly.

David

[rdb]

Thanks. I have some more questions though:

* Are transforms inside <Group> elements also global, or are they relative to the parent <Group> node?
* Joint transforms, are they global or relative?

Also, do you think this would be worth for inclusion into Panda3D, when it's stable? It just two directories that go into pandatool, and a few lines to ptloader. And of course references to fcollada in the ppremake files.

[drwr]

<Group> and <Joint> transforms are both relative to their parent. It's only vertices that are given in global space. I don't really know why; it's lost in the mists of time.

I'd love to have this become an official part of Panda.

David