                    DEFINING A MODEL TREE
              What to put in your Sources.pp files 
             to generate model and animation files

The model tree contains model and animation files, for instance in
MultiGen or SoftImage format, and contains rules for generating egg
files, creating character model and animation files, palettizing and
scaling textures, and creating final bam files for loading into an
application.

The egg format is Panda's working file format for storing general
models and animation files.  Typically, the build process for a given
model involves the following steps:

1. Convert from the source format (e.g. MultiGen or Maya, or any one
   of the other formats supported by command-line egg converters) into
   egg.

2. If the model represents an animated character or animation, run
   egg-optchar on the egg model file and its associated animation
   files to optimize animation performance.

3. Sometimes, additional filters are also run on the egg files, for
   instance to scale them or rotate them appropriately after
   conversion.

4. Run egg-palettize on the egg model file to optionally reduce the
   textures, and also pack the textures together onto palette images.
   The final textures are copied into the install directory at this
   point.

5. Run egg2bam to convert the final egg files to bam format and copy
   the resulting bam files into the install directory.


The ppremake system is set up to generate makefiles that automatically
perform these steps, although it currently generates only a Unix-like
makefile, so the at the moment a model tree can only be built on a
Linux or Unix machine, or on a PC with Cygwin installed.

See the dmodels tree in the Panda3D repository for an example of a
simple buildable model tree.  This is different from the models tree,
which is much simpler and includes only ready-to-load egg files; the
dmodels tree is set up as a full-fledged buildable model tree using
the ppremake system.  With a few small exceptions, the dmodels tree
does not contain any egg files, although intermediate egg files are
generated during the make process.  When you have finished building
the dmodels tree, you will have generated a number of bam files that
are fully optimized and ready to load for your current version of
Panda.


To set up a ppremake model tree, you must create a Sources.pp file in
the directory with the models (or in the directory above the SoftImage
tree in the case of SoftImage models).

The Sources.pp file for a directory in a model tree normally begins
with the line:

  #define DIR_TYPE models

which identifies this as a subdirectory in a model tree.  The rest of
the Sources.pp file contains blocks of the form:

#begin <build-type>
  #define SOURCES <source-filenames>
  ...
#end <build-type>

where <source-filenames> is a list of input filenames for this block,
and <build-type> defines the type of build rule and may be one of
flt_egg, maya_egg, maya_char_egg, install_egg, or a few others (the
complete list appears below).  Some kinds of build rules require other
parameters in addition to the list of source filenames.


The available build rules are:

flt_egg - Runs flt2egg to convert models from MultiGen (.flt) format to
  egg.

  SOURCES - a list of MultiGen files to convert.  These will have the
    extension .flt.  Each named MultiGen file will be converted to an
    egg file of the same name, with the extension .egg replacing .flt.

    We often use the syntax $[wildcard *.flt] in place of an explicit
    list of filenames to automatically pick up all the files with a
    .flt extension in the directory.

  FLT2EGG_OPTS - Specifies some additional command-line options that
    should be passed to flt2egg.  Common settings, for instance, might
    be -no to strip normals, or -uo ft to convert models to a standard
    scale in feet.  See flt2egg -h for a complete list of available
    options.  If you define this variable, it is usually better to
    extend it rather than replace its previous value, which allows
    global settings to be defined in the root Package.pp for the model
    tree.

  Example:
    #begin flt_egg
      #define SOURCES $[wildcard *.flt]
      #define FLT2EGG_OPTS $[FLT2EGG_OPTS] -no
    #end flt_egg

maya_egg - Runs maya2egg to convert models from Maya (.mb) format to
  egg.  This form of the block should be used to convert nonanimated
  models only; see maya_char_egg, below, to convert animated
  (character) models along with their animations.

  SOURCES - a list of Maya files to convert.  These will have the
    extension .mb or .ma.  Each Maya file will be converted to an egg
    file of the same name, with the extension .egg replacing .mb or
    .ma.

    As in flt_egg, above, you may use the syntax $[wildcard *.mb] in
    place of an explicit list of filenames to automatically pick up
    all the files with a .mb extension in the directory.

  MAYA2EGG_OPTS - As above, this specifies some additional
    command-line options that should be passed to maya2egg.  See
    maya2egg -h for a complete list of available options.

  Example:
    #begin maya_egg
      #define SOURCES $[wildcard *.mb]
      #define MAYA2EGG_OPTS $[MAYA2EGG_OPTS] -suppress-vcolor
    #end maya_egg

lwo_egg - As above, for Lightwave Object files (.lwo).

install_egg - Runs egg-palettize on the named egg files, then runs
  egg2bam to convert to bam file format, and then copies them into the
  install directory.

  SOURCES - a list of egg files to install.  Usually, these files are
    generated from a previous step (for instance, a previous flt_egg
    rule), but they may be ordinary egg files that are checked into
    the tree as source files.  Generally, you cannot use the wildcard
    syntax here (unlike in the flt_egg and maya_egg rules), because
    the egg files may not exist at the time ppremake is run; you must
    name each egg file explicitly (although it is possible to use the
    $[patsubst] function in ppremake to derive the egg files
    automatically from the names of the source files if you really
    want this).

  UNPAL_SOURCES - a list of egg files to install, omitting the
    egg-palettize step.  This is normally done for animation files
    that do not contain any geometry; there is no point in attempting
    to palettize these files since they contain no textures, and
    omitting the palettize step saves a bit of time building the model
    tree.  You should not name any egg files that include geometry or
    textures on the UNPAL_SOURCES line, it should only name animation
    files.

  SOURCE_DIR - if all of the source egg files can be found in some
    other directory than the current directory, this can optionally be
    defined to indicate that source directory (relative to the current
    directory).  Normally, this will only be the case when installing
    egg files that were generated from some intermediate step (for
    instance, a command specified by filter_egg; see filter_egg,
    below).  It is also possible to include the source directory as a
    part of each egg filename given in the SOURCES list, above.

  INSTALL_TO - the name of the install directory into which these
    models should be copied.  This is relative to the root of the
    tree, and may also be implicitly prefixed by phase_n/, where n is
    the phase number (see PHASE, below).  Since the install directory
    is usually the same for all files in a particular source
    directory, we usually define this variable globally at the top of
    the Sources.pp file; if it is defined there, it need not appear
    within the #begin .. #end scoping of install_egg.

  PHASE - the number of the phase directory into which these models
    should be copied.  This is optional, but if it is specified, it
    implicitly prefixes the INSTALL_TO directory with the name
    phase_n, where n is the phase number.  This is in support of
    building separate trees for a phased download.

    This phase number should match the phase number assigned to these
    egg files in the textures.txa file, so egg-palettize will know to
    install the textures into the same phase directory.

  Example:

    #begin install_egg
      #define SOURCES \
        trolley_station_DD.egg \
        DD_A1.egg DD_A2.egg DD_B1.egg DD_B2.egg DD_C1.egg DD_D1.egg \
        neighborhood_tunnel_DD.egg safe_zone_entrance_tunnel_DD.egg \
        safe_zone_tunnel_DD.egg DD_doors.egg DD_doors_practical.egg
      #define PHASE 6
    #end install_egg


maya_char_egg - Runs maya2egg to convert an animated model and its
  associated animation tables from a Maya file to a number of egg
  files.  Use this form of the block to convert animated files from
  Maya; use maya_egg (above) if you just have an unanimated model.

  To set up Maya animations, you should create multiple Maya files,
  one for each animation, and one more for the model itself.  Each
  Maya file should include the identical joint hierarchy; it is
  usually easiest to create the model file first and then duplicate it
  to create the animation files.  If you have multiple levels of
  detail for your character, you should create a separate Maya file
  for each LOD.

  The assumption is that your Maya files will follow the following
  naming convention: all Maya files that animate a particular
  character will begin with the same prefix, and end with the name of
  the animation, or with a name like "model" for the model file (or
  with the name of the particular level of detail, e.g. "1000" or
  "low", in the case of a multiple-LOD model).

  MAYA_PREFIX - Specifies the prefix that all of the Maya files have
    in common.

  EGG_PREFIX - Specifies an optional prefix to be given to any egg
    filenames generated by this rule.  We usually set this to the same
    as MAYA_PREFIX, to help associate the egg files with the Maya
    files they originated from.

  POLY_MODEL - Specifies the suffix that identifies the model file,
    e.g. "model" or "1000".  This is the form you will use if you
    intend to extract your model directly into polygons (whether it is
    modeled in NURBS or polygons).  The name of the Maya file that
    contains the model is $[MAYA_PREFIX]$[POLY_MODEL].mb (but see
    MODEL, below); it will be written to the egg file
    $[EGG_PREFIX]$[POLY_MODEL].egg.  If you have multiple levels of
    detail, you can only name one of them here; define a separate
    maya_char_egg block to get each of the other LOD's.

  NURBS_MODEL - Similar to the above; use this form if your character
    is modeled in NURBS, and you want to use an egg tool such as
    egg-qtess to convert NURBS to polygon models in a later step (for
    instance, to generate multiple levels of detail from one source
    file).

  MODEL - Optional.  If this is specified, it names the actual Maya
    file to use to extract the model.  That is, instead of
    $[MAYA_PREFIX]$[POLY_MODEL].mb, the model will be extracted from
    the Maya file $[MAYA_PREFIX]$[MODEL].mb.  It will still be written
    to the file $[EGG_PREFIX]$[POLY_MODEL].egg.  This can be used if
    you don't have a particular Maya file set aside to hold the model;
    in this case, you can name any one of the animation files, and the
    model will be pulled from that animation file.

  ANIMS - Specifies a list of animations to extract from Maya.  Each
    word in ANIMS represents a different animation; the name of the
    Maya file that contains the animation is derived from MAYA_PREFIX,
    e.g. $[MAYA_PREFIX]$[anim].mb.  Each anim is written to an egg
    file whose filename is derived from EGG_PREFIX,
    e.g. $[EGG_PREFIX]$[anim].egg.

  CHAR_NAME - Specifies the name of the character that is to be
    assigned to the models and animations generated in this rule.
    This name is written within each generated egg file, and should be
    different for each different character; it allows Panda to ensure
    at runtime that a given animation file is being played on the
    correct model file.
 
  MAYA2EGG_OPTS - Specifies some additional command-line options that
    are passed to maya2egg.  As in the maya_egg rule, above, you
    should extend rather than completely redefine this variable.

  Example:

    #begin maya_char_egg
      #define MAYA_PREFIX boss-
      #define EGG_PREFIX boss-torso-
      #define POLY_MODEL zero
      #define CHAR_NAME boss-torso
      #define ANIMS $[anims]
      #define MAYA2EGG_OPTS $[MAYA2EGG_OPTS] -subset body
    #end maya_char_egg


soft_char_egg - Runs soft2egg to convert a model or animation tables
  from a SoftImage 4.3 database to one or more egg files.

  DATABASE - Specifies the root directory of the SoftImage database.
    Often, this will be the same for all soft2egg rules in a
    particular directory, so we usually define this once at the top of
    the file instead of within each soft_char_egg rule.  This is
    usually a subdirectory within the current directory.

  SCENE_PREFIX - Specifies the name of the SoftImage scene; this is
    the prefix that SoftImage applies to each model and animation
    filename in the scene.

  The remaining options are similar to those in maya_char_egg, above.

  Example:
    
    #begin soft_char_egg
      #define SCENE_PREFIX suitA-
      #defer EGG_PREFIX suitA-
      #define POLY_MODEL lose-mod
      #define MODEL lose
      #define ANIMS lose
      #define CHAR_NAME suitA-lose
    #end soft_char_egg


    
optchar_egg - Runs egg-optchar on a character model (and/or multiple
  levels of detail of a single model) and all of the associated
  animations.  This step removes unnecessary joints for optimal
  runtime performance, exposes joints that are needed for show code
  operations, and can also perform other operations like scaling and
  rotating a character and its animation files together.

  It is important to run egg-optchar on the model together with all of
  its animation files at the same time; since egg-optchar operates the
  joint hierarchy, it must make any adjustments to all of the related
  animation files together.

  SOURCES - Specifies the list of egg files that are to be read.  This
    should include all models and animations that are associated the
    the same character.

  SOURCE_DIR - As with the install_egg rule, above, this optionally
    specifies the directory in which all of the egg files can be read,
    if it is not the current directory.

  TARGET_DIR - Specifies the subdirectory to which all of the
    resulting egg files should be written.

  OPTCHAR_OPTS - Specifies additional command-line arguments to pass
    to egg-optchar.  This normally includes rules to expose joints
    and/or transform models.  As with any of the OPTS variables, it is
    good practice to extend rather than completely redefine this
    variable.  Common options to egg-optchar are:

       -no                 - strip surface normals from the model.
       -expose joint-name  - expose joint-name to the show code.
       -flag polyset-name  - flag geometry for the show code.
       -TR x,y,z           - rotate about x, y, and z axes.
       -TS scale           - scale model by the indicated factor.

  Example:

    #begin optchar_egg
      #define TARGET_DIR optchar
      #define SOURCES \
        regexp/suitA-lose-mod.egg \
        suitA-lose.egg
      #define OPTCHAR_OPTS \
         $[OPTCHAR_OPTS] \
         -no -TR 0,180,0 -TS 0.25 \
         -expose joint-Rhold \
         -expose joint-Lhold -Dp joint-head \
         -expose joint-shadow \
         -expose joint-nameTag 
    #end optchar_egg
       

copy_egg - Copies an egg file, or a list of egg files, to another file
  with a different name.  This is particularly useful when extracting
  one file for multiple purposes, for instance before using
  egg-retarget-anim to retarget an animation to multiple skeletons.

  SOURCES - The list of original egg file names.

  TARGETS - The list of target egg file names.  There should be the
    same number of filenames in TARGETS as in SOURCES, and they will
    be matched up one-to-one; that is, the first egg file in SOURCES
    will be copied to the first filename in TARGET, the second egg
    file in SOURCES will be copied to the second filename in TARGET,
    and so on.

  Example:
    
    #begin copy_egg
      #define SOURCES elf-walk.egg elf-run.egg
      #define TARGETS dwarf-walk.egg dwarf-run.egg
    #end copy_egg


filter_egg - Runs an arbitrary egg command on one or more named egg
  files, one at a time.  This is usually done when an egg file
  generated by one of the above steps needs additional processing.
  The modified egg files are written into a specified directory to
  differentiate them from the source egg files; a later rule (for
  instance, optchar_egg or install_egg) should then specify to take
  its source egg files from this directory.

  SOURCES - The list of egg files to process.

  SOURCE_DIR - As above, the source directory in which to read the egg
    files, if it is not the current directory.

  TARGET_DIR - The subdirectory into which to write the generated egg
    files.

  COMMAND - The egg command to run.  This should include the string
    $[source], which will be replaced with the name of the source egg
    file, and $[target], which will be replaced with the target egg
    file.  Note that this variable should be defined with the #defer
    command instead of the #define command, to prevent the premature
    evaluation of $[target] and $[source].

  Example:
    
    #begin filter_egg
      #define TARGET_DIR scaled
      #define SOURCES \
        button.egg bluebox.egg
      #defer COMMAND \
        egg-trans -no -TR 0,180,0 -TS 0.25 \ 
        -o $[target] $[source]
    #end filter_egg


filter_char_egg - Similar to filter_egg, above, except that one
  command is run on all of the files named in SOURCES at the same
  time, instead of one at a time.  This is particularly useful for
  processing a character's model file and its animations all at the
  same time, hence the name of the rule.  This rule is most often used
  to invoke egg-topstrip, which is at the time of this writing the
  only egg tool other than egg-optchar that works on a model file and
  all animations at the same time.

  SOURCES - The list of egg files to process.

  SOURCE_DIR - As above, the source directory in which to read the egg
    files, if it is not the current directory.

  TARGET_DIR - The subdirectory into which to write the generated egg
    files.

  COMMAND - The egg command to run.  This should include the string
    $[sources], which will be replaced with the list of the source egg
    files, and it should probably include -d $[TARGET_DIR], to specify
    the output directory.  Note that this variable should be defined
    with the #defer command instead of the #define command, to prevent
    the premature evaluation of $[sources] and $[TARGET_DIR].

  Example:

    #begin filter_char_egg
      #define TARGET_DIR topstrip
      #define SOURCES \
        boss-torso-zero.egg \
        $[matrix boss-torso-,$[anims],.egg]
      #defer COMMAND \
        egg-topstrip -i -t joint_pelvis -d $[TARGET_DIR] $[sources]
    #end filter_char_egg


HOW TO POPULATE A CVS-CONTROLLED MODEL TREE

Since a model tree is really just a source tree like any other, you
can check in the source files--the .mb files or .flt files, for
instance, and all of the associated image files--as source files in a
CVS directory.  Then any developer can check out the source tree,
ppremake, and make install, exactly the same way you would build
Panda.  Of course, .mb files and .flt files are binary files, but CVS
(and most other revision control systems) allows you to add binary
files to the system.  You just can't merge multiple changes coming in
at once to the same file, so it's important to make frequent updates
and commits in order to minimize the chance of an accidental
collision with another artist.

In order for this to work, the source files must reference texture
paths and other external filenames using relative filenames--for
instance, a model should apply a texture named "../maps/grid.tif"
rather than "/home/drose/panda3d/dmodels/maps/grid.tif", for instance.
This is because the entire model tree might be checked out by another
developer within some other directory, and only relative paths within
the model tree will be valid.

For most modeling packages, it is not difficult to ensure that all of
the external references use only relative paths.  Populating a model
tree with, say, MultiGen source files may thus be as simple as copying
the MultiGen files and the image files in and then adding them to cvs
by hand (using -kb to indicate a binary file, of course).

Maya, on the other hand, for some reason insists on storing texture
references as full pathnames, even if you enter a relative path.  It
is not possible using the Maya interface to create an .mb file that
references its textures using relative pathnames.  This makes it
impossible to add a Maya file to a CVS repository in the normal way.

However, it *is* possible to store relative paths within a Maya file
if you use an OpenMaya program to generate the Maya file.  Panda
provides a utility called mayacopy that does this (among other useful
features).

In fact, Panda provides a family of utilities, with similar behavior;
at the time of this writing, the list includes mayacopy, fltcopy, and
lwocopy.  (There is also a separate tool called softcvs, used for
integrating SoftImage version 4.3 databases; this works differently
from the others, because of the very different nature of a SoftImage
scene.  The use of softcvs is not documented in this file.)

In general, the copy tools work like this:

  cd mymodels/src/mydir
  mayacopy /path/to/my/maya.mb

This copies the indicated Maya (or MultiGen, LightWave, etc.) file
from the named directory into the current directory.  It also copies
in all of the texture files referenced by the Maya file, and it
modifies the Maya file to use only local pathnames to the newly copied
texture files, instead of the full pathnames that would have been
stored in the original Maya file.

A little heuristic is used to decide where to place each texture image
referenced by the Maya file.  The following rules are applied:

  (1) If another texture file with the same name already exists
      elsewhere in the source hierarchy, assume this texture image
      represents the same image (or a new version of that image) and
      overwrite it.  (Note that it is not a good idea to have
      different texture images that are stored under the same
      filename.  egg-palettize also makes the assumption that any two
      files with the same filename represent the same texture image.)

  (2) If the texture file does not already exist, copy it into src/maps.

  (3) If the src/maps directory does not exist, copy the texture file
      into the same directory with the source Maya file.

As the Maya file and each texture file is copied into the source tree,
the cvs command is automatically invoked to add the file(s) to the
repository, if necessary.  It is still your responsibility to issue
the cvs commit command when you are ready to make the changes
permanent.

You can use mayacopy, fltcopy, etc. to copy in a file from a
completely different hierarchy, or to update a file already within the
source tree.  It is particularly important to re-run mayacopy after
modifying any .mb file using Maya, since Maya will replace all of the
local pathnames with full pathnames each time you save the file.
Re-running mayacopy on the same file, in place, will restore the
relative pathnames again.



