35 bool kill_cmd =
false;
37 bool compress_flag =
false;
38 int default_compression_level = 6;
40 bool got_multifile_name =
false;
41 bool to_stdout =
false;
42 bool encryption_flag =
false;
44 bool got_password =
false;
46 bool got_header_prefix =
false;
48 bool got_chdir_to =
false;
49 size_t scale_factor = 0;
52 vector_string sign_params;
55 string dont_compress_str =
"jpg,png,mp3,ogg";
58 string text_ext_str =
"txt";
60 time_t source_date_epoch = (time_t)-1;
61 bool got_record_timestamp_flag =
false;
62 bool record_timestamp_flag =
true;
75 const char *nptr = str.c_str();
77 int result = strtol(nptr, &endptr, 10);
97 "Usage: multify -[c|r|u|t|x] -f <multifile_name> [options] <subfile_name> ...\n";
104 "multify is used to store and extract files from a Panda Multifile.\n"
105 "This is similar to a tar or zip file in that it is an archive file that\n"
106 "contains a number of subfiles that may later be extracted.\n\n"
108 "Panda's VirtualFileSystem is capable of mounting Multifiles for direct\n"
109 "access to the subfiles contained within without having to extract them\n"
110 "out to independent files first.\n\n"
112 "The command-line options for multify are designed to be similar to those\n"
113 "for tar, the traditional Unix archiver utility.\n\n"
117 " You must specify exactly one of the following command switches:\n\n"
120 " Create a new Multifile archive. Subfiles named on the command line\n"
121 " will be added to the new Multifile. If the Multifile already exists,\n"
122 " it is first removed.\n\n"
125 " Rewrite an existing Multifile archive. Subfiles named on the command\n"
126 " line will be added to the Multifile or will replace subfiles within\n"
127 " the Multifile with the same name. The Multifile will be repacked\n"
128 " after completion, even if no Subfiles were added.\n\n"
131 " Update an existing Multifile archive. This is similar to -r, except\n"
132 " that files are compared byte-for-byte with their corresponding files\n"
133 " in the archive first. If they have not changed, the multifile is not\n"
134 " modified (other than to repack it if necessary).\n\n"
137 " List the contents of an existing Multifile. With -v, this shows\n"
138 " the size of each Subfile and its compression ratio, if compressed.\n\n"
141 " Extract the contents of an existing Multifile. The Subfiles named on\n"
142 " the command line, or all Subfiles if nothing is named, are extracted\n"
143 " into the current directory or into whichever directory is specified\n"
147 " Delete (kill) the named Subfiles from the Multifile. The Multifile\n"
148 " will be repacked after completion.\n\n"
151 " You must always specify the following switch:\n\n"
153 " -f <multifile_name>\n"
154 " Names the Multifile that will be operated on.\n\n\n"
156 " The remaining switches are optional:\n\n"
159 " Run verbosely. In -c, -r, or -x mode, list each file as it is\n"
160 " written or extracted. In -t mode, list more information about each\n"
164 " Compress subfiles as they are written to the Multifile. Unlike tar\n"
165 " (but like zip), subfiles are compressed individually, instead of the\n"
166 " entire archive being compressed as one stream. It is not necessary\n"
167 " to specify -z when extracting compressed subfiles; they will always be\n"
168 " decompressed automatically. Also see -Z, which restricts which\n"
169 " subfiles will be compressed based on the filename extension.\n\n"
172 " Encrypt subfiles as they are written to the Multifile using the password\n"
173 " specified with -p, below. Subfiles are encrypted individually, rather\n"
174 " than encrypting the entire multifile, and different subfiles may be\n"
175 " encrypted using different passwords (although this requires running\n"
176 " multify multiple times). It is not possible to encrypt the multifile's\n"
177 " table of contents using this interface, but see the pencrypt program to\n"
178 " encrypt the entire multifile after it has been generated.\n\n"
182 " Specifies the password to encrypt or decrypt subfiles. If this is not\n"
183 " specified, and passwords are required, the user will be prompted from\n"
184 " standard input.\n\n"
187 " Specifies a header_prefix to write to the beginning of the multifile.\n"
188 " This is primarily useful for creating a multifile that can be invoked\n"
189 " directly as a program from the shell on Unix-like environments,\n"
190 " for instance, p3d files. The header_prefix must begin with a hash\n"
191 " mark and end with a newline; this will be enforced if it is not\n"
192 " already so. This only has effect in conjunction with with -c, -u,\n"
195 " -F <scale_factor>\n"
196 " Specify a Multifile scale factor. This is only necessary to support\n"
197 " Multifiles that will exceed 4GB in size. The default scale factor is\n"
198 " 1, which should be sufficient for almost any application, but the total\n"
199 " size of the Multifile will be limited to 4GB * scale_factor. The size\n"
200 " of individual subfiles may not exceed 4GB in any case.\n\n"
202 " -C <extract_dir>\n"
204 " Change to the named directory before working on files;\n"
205 " that is, extraction/creation/update and replace will be based on this path\n\n"
208 " With -x, extract subfiles to standard output instead of to disk.\n\n"
209 " -Z <extension_list>\n"
210 " Specify a comma-separated list of filename extensions that represent\n"
211 " files that are not to be compressed. The default if this is omitted is\n"
212 " \"" << dont_compress_str <<
"\". Specify -Z \"\" (be sure to include the space) to allow\n"
213 " all files to be compressed.\n\n"
214 " -X <extension_list>\n"
215 " Specify a comma-separated list of filename extensions that represent\n"
216 " text files. These files are opened and read in text mode, and added to\n"
217 " the multifile with the text flag set. The default if this is omitted is\n"
218 " \"" << text_ext_str <<
"\". Specify -X \"\" (be sure to include the space) to record\n"
219 " all files in binary mode.\n\n"
222 " Enable or disable the recording of file timestamps within the multifile.\n"
223 " If <flag> is 1, timestamps will be recorded within the multifile for\n"
224 " each subfile added; this is the default behavior. If <flag> is 0,\n"
225 " timestamps will not be recorded, which will make it easier to do a\n"
226 " bitwise comparison between multifiles to determine whether their\n"
227 " contents are equivalent.\n\n"
230 " Specify the compression level when -z is in effect. Larger numbers\n"
231 " generate slightly smaller files, but compression takes longer. The\n"
232 " default is -" << default_compression_level <<
".\n\n"
234 " -S file.crt[,chain.crt[,file.key[,\"password\"]]]\n"
235 " Sign the multifile. The signing certificate should be in PEM form in\n"
236 " file.crt, with its private key in PEM form in file.key. If the key\n"
237 " is encrypted on-disk, specify the decryption password as the third\n"
238 " option. If a certificate chain is required, chain.crt should also\n"
239 " be specified; note that the separating commas should be supplied\n"
240 " even if this optional filename is omitted.\n"
241 " You may also provide a single composite file that contains the\n"
242 " certificate, chain, and key in the same file.\n"
243 " PEM form is the form accepted by the Apache web server. The\n"
244 " signature is written to the multifile to prove it is unchanged; any\n"
245 " subsequent change to the multifile will invalidate the signature.\n"
246 " This parameter may be repeated to sign the multifile with additional\n"
247 " certificates.\n\n";
253 cerr <<
"Enter password: ";
254 std::getline(std::cin, password);
263 is_named(
const string &subfile_name,
const vector_string ¶ms) {
266 if (params.empty()) {
271 vector_string::const_iterator pi;
272 for (pi = params.begin(); pi != params.end(); ++pi) {
273 if (subfile_name == (*pi)) {
282 is_text(
const Filename &subfile_name) {
287 if (text_ext.find(ext) != text_ext.end()) {
296 get_compression_level(
const Filename &subfile_name) {
298 if (!compress_flag) {
304 if (dont_compress.find(ext) != dont_compress.end()) {
310 return default_compression_level;
320 cerr <<
"Unable to scan directory " << directory_name <<
"\n";
325 filenames.reserve(files.size());
326 vector_string::const_iterator fi;
327 for (fi = files.begin(); fi != files.end(); ++fi) {
328 Filename subfile_name(directory_name, (*fi));
329 filenames.push_back(subfile_name);
332 return do_add_files(multifile, filenames);
339 for (fi = filenames.begin(); fi != filenames.end(); ++fi) {
343 if (!do_add_directory(multifile, subfile_name)) {
347 }
else if (!subfile_name.
exists()) {
348 cerr <<
"Not found: " << subfile_name <<
"\n";
352 if (is_text(subfile_name)) {
358 string new_subfile_name;
361 (subfile_name, subfile_name, get_compression_level(subfile_name));
364 (subfile_name, subfile_name, get_compression_level(subfile_name));
366 if (new_subfile_name.empty()) {
367 cerr <<
"Unable to add " << subfile_name <<
".\n";
371 cout << new_subfile_name <<
"\n";
380 add_files(
const vector_string ¶ms) {
382 if (append || update) {
384 cerr <<
"Unable to open " << multifile_name <<
" for updating.\n";
389 cerr <<
"Unable to open " << multifile_name <<
" for writing.\n";
394 if (got_record_timestamp_flag) {
398 if (encryption_flag) {
403 if (got_header_prefix) {
408 cerr <<
"Setting scale factor to " << scale_factor <<
"\n";
413 filenames.reserve(params.size());
414 vector_string::const_iterator si;
415 for (si = params.begin(); si != params.end(); ++si) {
417 filenames.push_back(subfile_name);
421 if (got_chdir_to && !chdir_to.
chdir()) {
422 cout <<
"Failed to chdir to " << chdir_to <<
": " << strerror(errno) << endl;
426 bool okflag = do_add_files(multifile, filenames);
441 if (!multifile->
repack()) {
442 cerr <<
"Failed to write " << multifile_name <<
".\n";
446 if (!multifile->
flush()) {
447 cerr <<
"Failed to write " << multifile_name <<
".\n";
456 extract_files(
const vector_string ¶ms) {
457 if (!multifile_name.
exists()) {
458 cerr << multifile_name <<
" not found.\n";
462 if (!multifile->open_read(multifile_name)) {
463 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
473 bool any_encrypted =
false;
474 for (i = 0; i < num_subfiles && !any_encrypted; i++) {
476 if (is_named(subfile_name, params)) {
478 any_encrypted =
true;
488 for (i = 0; i < num_subfiles; i++) {
490 if (is_named(subfile_name, params)) {
493 filename =
Filename(chdir_to, subfile_name);
497 cerr << filename <<
"\n";
502 cout << filename <<
"\n";
513 kill_files(
const vector_string ¶ms) {
514 if (!multifile_name.
exists()) {
515 cerr << multifile_name <<
" not found.\n";
520 cerr <<
"Unable to open " << multifile_name <<
" for read/write.\n";
524 if (got_header_prefix) {
529 while (i < multifile->get_num_subfiles()) {
531 if (is_named(subfile_name, params)) {
535 cout << filename <<
"\n";
552 if (!multifile->
repack()) {
553 cerr <<
"Failed to write " << multifile_name <<
".\n";
557 if (!multifile->
flush()) {
558 cerr <<
"Failed to write " << multifile_name <<
".\n";
569 cerr <<
"Cannot sign multifiles without OpenSSL compiled in.\n";
577 cerr <<
"Unable to re-open " << multifile_name <<
" for signing.\n";
581 vector_string::iterator si;
582 for (si = sign_params.begin(); si != sign_params.end(); ++si) {
583 const string ¶m = (*si);
587 size_t comma1 = param.find(
',');
588 if (comma1 == string::npos) {
592 size_t comma2 = param.find(
',', comma1 + 1);
593 if (comma2 == string::npos) {
597 size_t comma3 = param.find(
',', comma2 + 1);
598 if (comma3 == string::npos) {
602 password = param.substr(comma3 + 1);
607 if (!multifile->add_signature(certificate, chain, pkey, password)) {
617 format_timestamp(
bool record_timestamp, time_t timestamp) {
618 static const size_t buffer_size = 512;
619 static char buffer[buffer_size];
621 if (!record_timestamp) {
626 if (timestamp == 0) {
628 return " (no date) ";
631 time_t now = time(
nullptr);
632 struct tm *tm_p = localtime(×tamp);
634 if (timestamp > now || (now - timestamp > 86400 * 365)) {
637 strftime(buffer, buffer_size,
"%b %d %Y", tm_p);
640 strftime(buffer, buffer_size,
"%b %d %H:%M", tm_p);
647 list_files(
const vector_string ¶ms) {
648 if (!multifile_name.
exists()) {
649 cerr << multifile_name <<
" not found.\n";
656 if (istr ==
nullptr) {
657 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
662 if (!multifile->open_read(
new IStreamWrapper(istr,
true),
true)) {
663 cerr <<
"Unable to open " << multifile_name <<
" for reading.\n";
671 cout << num_subfiles <<
" subfiles:\n" << std::flush;
672 for (i = 0; i < num_subfiles; i++) {
674 if (is_named(subfile_name, params)) {
675 char encrypted_symbol =
' ';
677 encrypted_symbol =
'e';
679 char text_symbol =
' ';
687 if (orig_length != 0) {
688 ratio = (double)internal_length / (
double)orig_length;
691 printf(
"%12d worse %c%c %s %s\n",
693 encrypted_symbol, text_symbol,
696 subfile_name.c_str());
698 printf(
"%12d %3.0f%% %c%c %s %s\n",
700 100.0 - ratio * 100.0,
701 encrypted_symbol, text_symbol,
704 subfile_name.c_str());
707 printf(
"%12d %c%c %s %s\n",
709 encrypted_symbol, text_symbol,
712 subfile_name.c_str());
719 cout <<
"Last modification "
720 << format_timestamp(
true, multifile->
get_timestamp()) <<
"\n";
727 cout <<
"Multifile needs to be repacked.\n";
730 for (i = 0; i < num_subfiles; i++) {
732 if (is_named(subfile_name, params)) {
733 cout << subfile_name <<
"\n";
739 int num_signatures = multifile->get_num_signatures();
740 if (num_signatures != 0) {
742 for (i = 0; i < num_signatures; ++i) {
743 cout <<
"Signed by " << multifile->get_signature_friendly_name(i);
744 int verify_result = multifile->validate_signature_certificate(i);
745 if (verify_result == 0) {
746 cout <<
" (certificate validated)\n";
748 cout <<
" (certificate unknown, reason " << verify_result <<
")\n";
751 multifile->write_signature_certificate(i, cout);
762 tokenize_extensions(
const string &str,
pset<string> &extensions) {
764 while (p < str.length()) {
765 size_t q = str.find_first_of(
",", p);
766 if (q == string::npos) {
767 extensions.insert(str.substr(p));
770 extensions.insert(str.substr(p, q - p));
773 extensions.insert(
string());
777 main(
int argc,
char **argv) {
787 if (*argv[1] !=
'-' && *argv[1] !=
'\0') {
788 char *new_arg = (
char *)PANDA_MALLOC_ARRAY(strlen(argv[1]) + 2);
790 strcpy(new_arg + 1, argv[1]);
795 const char *source_date_epoch_str = getenv(
"SOURCE_DATE_EPOCH");
796 if (source_date_epoch_str !=
nullptr && source_date_epoch_str[0] != 0) {
797 source_date_epoch = (time_t)strtoll(source_date_epoch_str,
nullptr, 10);
802 static const char *optflags =
"crutxkvz123456789Z:T:X:S:f:OC:ep:P:F:h";
803 int flag = getopt(argc, argv, optflags);
805 while (flag != EOF) {
829 compress_flag =
true;
832 default_compression_level = 1;
833 compress_flag =
true;
836 default_compression_level = 2;
837 compress_flag =
true;
840 default_compression_level = 3;
841 compress_flag =
true;
844 default_compression_level = 4;
845 compress_flag =
true;
848 default_compression_level = 5;
849 compress_flag =
true;
852 default_compression_level = 6;
853 compress_flag =
true;
856 default_compression_level = 7;
857 compress_flag =
true;
860 default_compression_level = 8;
861 compress_flag =
true;
864 default_compression_level = 9;
865 compress_flag =
true;
868 dont_compress_str = optarg;
871 text_ext_str = optarg;
874 sign_params.push_back(optarg);
880 (flag != 0 && flag != 1)) {
881 cerr <<
"Invalid timestamp flag: " << optarg <<
"\n";
885 record_timestamp_flag = (flag != 0);
886 got_record_timestamp_flag =
true;
891 got_multifile_name =
true;
901 encryption_flag =
true;
908 header_prefix = optarg;
909 got_header_prefix =
true;
914 scale_factor = strtol(optarg, &endptr, 10);
915 if (*endptr !=
'\0') {
916 cerr <<
"Invalid integer: " << optarg <<
"\n";
920 if (scale_factor == 0) {
921 cerr <<
"Scale factor must be nonzero.\n";
935 cerr <<
"Unhandled switch: " << flag << endl;
938 flag = getopt(argc, argv, optflags);
940 argc -= (optind - 1);
941 argv += (optind - 1);
944 if ((create?1:0) + (append?1:0) + (update?1:0) + (tlist?1:0) + (extract?1:0) + (kill_cmd?1:0) != 1) {
945 cerr <<
"Exactly one of -c, -r, -u, -t, -x, -k must be specified.\n";
950 if (!got_multifile_name) {
951 cerr <<
"Multifile name not specified.\n";
957 tokenize_extensions(dont_compress_str, dont_compress);
960 tokenize_extensions(text_ext_str, text_ext);
963 vector_string params;
964 params.reserve(argc - 1);
965 for (
int i = 1; i < argc; i++) {
966 params.push_back(argv[i]);
970 if (create || append || update) {
971 okflag = add_files(params);
972 }
else if (extract) {
973 if (got_record_timestamp_flag) {
974 cerr <<
"Warning: -T ignored on extract.\n";
976 okflag = extract_files(params);
977 }
else if (kill_cmd) {
978 if (got_record_timestamp_flag) {
979 cerr <<
"Warning: -T ignored on kill.\n";
981 okflag = kill_files(params);
983 if (got_record_timestamp_flag) {
984 cerr <<
"Warning: -T ignored on list.\n";
986 okflag = list_files(params);
989 if (okflag && !sign_params.empty()) {
The name of a file, such as a texture file or an Egg file.
bool scan_directory(vector_string &contents) const
Attempts to open the named filename as if it were a directory and looks for the non-hidden files with...
void set_binary()
Indicates that the filename represents a binary file.
bool chdir() const
Changes directory to the specified location.
void set_text()
Indicates that the filename represents a text file.
static Filename from_os_specific(const std::string &os_specific, Type type=T_general)
This named constructor returns a Panda-style filename (that is, using forward slashes,...
bool is_directory() const
Returns true if the filename exists and is a directory name, false otherwise.
std::string get_extension() const
Returns the file extension.
bool exists() const
Returns true if the filename exists on the disk, false otherwise.
This class provides a locking wrapper around an arbitrary istream pointer.
A file that contains a set of files.
void set_header_prefix(const std::string &header_prefix)
Sets the string which is written to the Multifile before the Multifile header.
size_t get_scale_factor() const
Returns the internal scale factor for this Multifile.
bool is_subfile_text(int index) const
Returns true if the indicated subfile represents text data, or false if it represents binary data.
bool open_read_write(const Filename &multifile_name)
Opens the named Multifile on disk for reading and writing.
std::string update_subfile(const std::string &subfile_name, const Filename &filename, int compression_level)
Adds a file on disk to the subfile.
bool extract_subfile_to(int index, std::ostream &out)
Extracts the nth subfile to the indicated ostream.
get_num_subfiles
Returns the number of subfiles within the Multifile.
void set_record_timestamp(bool record_timestamp)
Sets the flag indicating whether timestamps should be recorded within the Multifile or not.
bool flush()
Writes all contents of the Multifile to disk.
bool is_subfile_compressed(int index) const
Returns true if the indicated subfile has been compressed when stored within the archive,...
time_t get_subfile_timestamp(int index) const
Returns the modification time of the nth subfile.
bool get_record_timestamp() const
Returns the flag indicating whether timestamps should be recorded within the Multifile or not.
void set_scale_factor(size_t scale_factor)
Changes the internal scale factor for this Multifile.
void set_timestamp(time_t timestamp)
Changes the overall mudification timestamp of the multifile.
size_t get_subfile_length(int index) const
Returns the uncompressed data length of the nth subfile.
void set_encryption_flag(bool flag)
Sets the flag indicating whether subsequently-added subfiles should be encrypted before writing them ...
bool is_subfile_encrypted(int index) const
Returns true if the indicated subfile has been encrypted when stored within the archive,...
time_t get_timestamp() const
Returns the modification timestamp of the overall Multifile.
void set_encryption_password(const std::string &encryption_password)
Specifies the password that will be used to encrypt subfiles subsequently added to the multifile,...
bool needs_repack() const
Returns true if the Multifile index is suboptimal and should be repacked.
bool repack()
Forces a complete rewrite of the Multifile and all of its contents, so that its index will appear at ...
bool open_write(const Filename &multifile_name)
Opens the named Multifile on disk for writing.
size_t get_subfile_internal_length(int index) const
Returns the number of bytes the indicated subfile consumes within the archive.
void remove_subfile(int index)
Removes the nth subfile from the Multifile.
get_subfile_name
Returns the name of the nth subfile.
std::string add_subfile(const std::string &subfile_name, const Filename &filename, int compression_level)
Adds a file on disk as a subfile to the Multifile.
bool extract_subfile(int index, const Filename &filename)
Extracts the nth subfile into a file with the given name.
A hierarchy of directories and files that appears to be one continuous file system,...
std::istream * open_read_file(const Filename &filename, bool auto_unwrap) const
Convenience function; returns a newly allocated istream if the file exists and can be read,...
static VirtualFileSystem * get_global_ptr()
Returns the default global VirtualFileSystem.
This is our own Panda specialization on the default STL set.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
void preprocess_argv(int &argc, char **&argv)
Processes the argc, argv pair as needed before passing it to getopt().
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
int string_to_int(const string &str, string &tail)
A string-interface wrapper around the C library strtol().
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.
PANDA 3D SOFTWARE Copyright (c) Carnegie Mellon University.