3DS Exporter

Let me say first: I am not asking if this is fixed. Actually, my team strongly desires to use max, so I’m fixing it up. The initial thing is to stabalize it so it stops crashing.

I attached OllyDbg to 3DS, and waited for it to crash. Some things of note: First, it doesn’t crash reliably every time. Even if I do identical things every time. Further sometimes I can’t seem to force it to crash…

Second: this is the call trace at the time of the crash:

MSVCP80.?_Tidy@?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@IAEX_NI@Z                                             
MSVCP80.??1?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@QAE@XZ                                                    
maxegg10.ProgramBase::Option::~Option                                                                                             
? maxegg10.std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramBase::Option  
? maxegg10.std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramBase::Option  
? maxegg10.std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramBase::Option  
? maxegg10.std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramBase::Option  
? maxegg10.std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,ProgramBase::Option  
maxegg10.ProgramBase::~ProgramBase
maxegg10.EggBase::~EggBase
maxegg10.EggSingleBase::~EggSingleBase
Includes maxegg10.0E9E8523
maxegg10.MaxEggExpOptions::DoExport

The Lowest Level call there is ProgramBase::Option::~Option. This function isn’t explictly defined, take a look at the declaration:

  class Option {
  public:
    string _option;
    string _parm_name;
    int _index_group;
    int _sequence;
    string _description;
    OptionDispatchFunction _option_function;
    OptionDispatchMethod _option_method;
    bool *_bool_var;
    void *_option_data;
  };

Since its not explicitly defined, all it can be doing is deconstructing stack members. So we’re looking at _option, _param_name, _description, etc. The problem is that these are stable in all other cases, so I’m guessing that what we’re seeing is a buffer overflow. Anyone have any suggestions as to where to look?

I’m going to throw a memory profiler at it. Set up some fencing, and hope to see results… If anyone has suggestions, or feature requests, let me know.

Okay, I can’t apply my normal memory management header since every class is based on a memory overriding solution… Which I actually like. I’ve been using a variation of Paul Nettle’s memory manager from the flipcode days, but this method has plenty of interesting merit.

The first time I looked into this, it looked like it was crashing in the deconstructor. I noticed the following code:

  // This was put in try block because originally deleting pmteConverter 
  // would throw an exception.  That no longer happens, but this is still
  // here for good measure
  delete pmteConverter; 

This is clearly NOT in a try block. It really wouldn’t matter though, in most cases. An access violation is an asynchronous exception. The only way to get it to be caught is via the compiler switch ‘/EHa’ (for msvc anyway… not sure if this is the same for the platform tools). The problem is this only hides the problem, it doesn’t solve it.

So… my experience with console debugging is that when your tools don’t help you, force the code to.

I converted:

 pmteConverter->parse_command_line(iParameterCount, apcParameters);
  if (expWholeScene) pmteConverter->Run(NULL, 0);
  else pmteConverter->Run(nodeList, numNodes);

  successful = pmteConverter->IsSuccessful();

to

 pmteConverter->parse_command_line(iParameterCount, apcParameters);
//  if (expWholeScene) pmteConverter->Run(NULL, 0);
//  else pmteConverter->Run(nodeList, numNodes);

  successful = true;//pmteConverter->IsSuccessful();

Since I was assuming the error was going to be in the run cycle. Apparently I was mistaken.

Wait… you’re saying it crashes even if you take the entire Run call out?

Interesting.

By the way, I concur on the buffer overrun. I’ve debugged this code myself using visual studio and it always crashes in some sort of argument-parsing routine, but there’s nothing wrong with the argument-parsing code.

Sorry about the delay on this, I was really expecting to have more time last week.

You should get a kick out of this Josh. I went through the process of removing everything possible, and trying to isolate the source in the argument processing. Suffice to say, I wasn’t getting anywhere. Assuming that nothing is safe, I physically removed the logging. Things are suddenly working safely. I reverted back to the original and removed ONLY LOGGING, and its working fine. I’m going to spend some more time to isolate the actual problem, but it appears that its within the logging setup.

Alright, this isn’t the direct problem, though it is of note:

myHierarchyLevel = myWrittenHierarchyLevel;
sprintf( beginMsg, "#BEGIN {%s}", *hierarchyDepthIterator );
WriteToPipe( beginMsg );
myHierarchyLevel = tempHierarchyLevel;

The problem this is a concern, is because beginMsg has a maxLen of 64. This COULD be ample, but just as an example:

is one of the items that gets pushed through that. The string is 52 characters by itself, plus the 9 from the formatting. You get 61. Just dangerously close.

I’m having a hard time debugging this, because I can’t make it crash reliably (I keep finding myself saying: Why won’t it crash!!)

If anyone knows a procedure to cause it to crash, or has a file that it crashes reliably on, let me know.

I’ve got a for loop running 500 times per export… and still, its not reliably crashing within that 500 times. I’ve also noticed that adding data to the stack makes it less likely to crash… At this point I may just disable the logging system, as it still seems like thats the cause of things.

Josh, I know you said it looked like the parser, but I’m not seeing it. It could be that when I’m disabling the logging, that I’m removing memory portions that we doing the over-writes, or being over-written. The problem is: it simply works without the logging enabled… and I can’t see why. I’m fine pulling the plug on the logging, but I’m afraid that may not be it, and this may happen again later.