prevnext   » SZS: Wiimms SZS Toolset » wlect: Wiimms LE-CODE Tool » wlect distribution

wlect distribution

This command manages data for LE-CODE track distributions. It reads any number of source files with different file types, collects the data and creates any number of files with different file types.

Use the command without keyword to get an extended description.

Contents

1.   Syntax

wlect DISTRIBUTION [argument]...

2.   Options

Options
Option Param Description
--load-prefix file Read in a PREFIX file and replace the internal prefix database with the content. https://ct.wiimm.de/export/prefix is the authoritative source for this.
--plus chars If a name begins with a plus sign, then all characters up to the first space are recognized as a plus prefix. The first part consists of all the plus signs followed by any other characters. The first character of the second part is used to determine the sort value. If the second part is empty, then the last plus sign is used instead.

The higher sorting value is now calculated from the number of plus signs in the first part, the more plus signs, the smaller the value. The first character from the second part determines the lower value. If it is in parameter CHARS then the position determines the value, otherwise the ASCII value to which 500 is added.

--track-dir directory Define a destination directory, where track files are copied, moved or linked to. Usually it is .../Race/Course/.

Files are searched in directories defined by --copy-tracks, --move-tracks, --move1-tracks and --link-tracks in definition order. Already existing files are removed before operation.

--copy-tracks directory Define a search directory for all included tracks. Files found in this directory are copied to the directory specified by option --track-dir.
--move-tracks directory Define a search directory for all included tracks. Files found in this directory are moved to the directory specified by option --track-dir. If moving fails, the file is copied and the source removed.
--move1-tracks directory Define a search directory for all included tracks. Files found in this directory are moved to the directory specified by option --track-dir, but only if a file has not more than one hard link. This guarantees an unique version of the file. If the file has more hard links or moving fails, the file is copied and the source removed.
--szs-mode mode If a track is inserted via the SZS file, then the associated directory is automatically included in the list of search directories for option --track-dir. This option now sets the transfer mode. OFF turns this feature off. The values COPY, MOVE, MOVE1 and LINK define one of the 4 modes. Default mode is LINK.
-i --ignore Ignore non existing source files without warning.
-H --no-header Suppress the syntax information section in LE text files.
-B --brief Suppress cross reference comments.
-X --export Enable the export mode and create small and machine readable text files for easy post processing.
-d --dest path Define a destination path (directory/file). The destination - means: write to standard output.

The path may contain escape sequences: %Q is replaced by the fully qualified source name. %P and %F are replaced by the source path or by the filename. %N and %E are replaced by source filename without extension or by the source extension. Finally, %T is replaced by the default extension of the destination format.

A '?' direct behind '%' in %E and %T conversions avoid that the same extension is used twice in row.

-D --DEST path Like --dest, but create the directory path automatically.
-E --esc char Define an alternative escape character for destination files. The default is '%'. For Windows batch files it is a good choice to set '-E$'.
-o --overwrite Overwrite already existing files without warning and ignore option --number.
--number If a file already exist, append a number directly before the file extension to make the filename unique. If other numbered files already exist (ignoring case), use the maximum existing index+1. --num is a short cut.
-r --remove-dest Remove already existing files before creating it. If set, --overwrite is ignored. --rm-dest is a short cut.
-u --update Update only existing files and don't create new files. If set, --overwrite and --remove-dest are ignored.
-p --preserve Preserve file times (atime+mtime) while converting or copying files.

3.   Description

This new command was created to handle all aspects related to LE-CODE based distributions. To do this, it reads in any number of sources with different file formats to create a holistic model. Any number of files with different file formats can then be generated from this model. It's all controlled by processing options. After writing some files, other files can also be loaded, e.g. with alternative names, in order to then generate alternative output files.

In order to make all this possible, a sometimes very long command line is entered. This is then processed step by step from left to right. Each argument is either a processing option, an instruction (e.g. write a file) or a filename to read a file.

This command replaces commands like wlect patch ... to patch a LE-CODE binary file or wszst distribution ... to prepare a file for ct.wiimm.de.

For more details read the built-in help.

4.   Built-in Help

Below is a copy of the built-in help which can be displayed by wlect distribution or shorter by wlect dis as colored text. It is also available as text file.

Command »wlect DISTRIBUTION«

  This command manages data for LE-CODE track distributions. It reads any number of
  source files with different file types, collects the data and creates any number of
  files with different file types.

  »wlect DISTRIBUTION« is a very powerful tool when dealing with distributions. Because
  of the complexity, the input line offers many possibilities. First, the default options
  (arguments beginning with 1 or 2 minus signs like -B or --brief) are evaluated. Then
  the remaining arguments of the command line are evaluated step by step from left to
  right. There are input files that change the internal model. The instructions are used
  to change the data or to write data to files. The processing options affect both
  reading and writing.

  This entire command is still under development. Errors are quite likely!

  Syntax:

    wlect DISTRIBUTION
    wlect DISTRIBUTION option... argument...

  »DIS« and »DISTRIB« are well defined shortcuts for »DISTRIBUTION«. Without arguments,
  this help is printed.


Arguments:

  Arguments are divided into 3 groups: processing options, instructions and filenames.

  If an argument beginns with a plus sign (+), then is is scanned as a comma-separated
  list of processing options. To use a filename that starts with a plus sign (e.g.
  »+file.txt«), precede it with »./« (e.g. »./+file.txt«).

  Otherwise, if the argument is of the form COMMAND=PARAMETER, then it is a instruction.
  COMMAND is a keyword (case-insenitive) and starts with a letter followed by any number
  of letters, digits and minus signs. PARAMETER must consist of at least one character.
  If PARAMETER is empty, then the next argument is used as PARAMETER. This also allows
  the syntax COMMAND= PARAMETER (2 arguments), which is helpful for the automatic
  completion of filenames.

  Otherwise it is a filename of an input file. The file is read and generally overwrites
  existing content. This means that the file last read in has the highest priority.
  However, this can be influenced by the processing options. Arguments beginning with »/«
  or »./« are always recognized as filenames.

  All arguments are executed in the order in which they were entered without any logging.
  Only error messages are displayed. With option --verbose (short: -v),  at least one log
  line for each argument is printed.

  It is possible to store arguments into a file (e.g. into file »param.txt«) and to
  include those file by option -@FILENAME (e.g. by »-@param.txt«). This option can be
  used multiple times and is evaluated before the actual analysis of the command line.
  Command »wlect argtest ...« is suitable for tests.


Input Files:

  Input files are processed in the order in which they were entered. They supplement or
  overwrite the internal data structure with its diverse data.

  If a file does not exist but the associated filename contains wildcards (any of »*?[«),
  then all matching files are loaded as source. So it's possible to keep the command line
  small if using »'*.szs'« (with apostrophs) instead of »*.szs«.

  The following file types are recognized and processed. The names used here are the same
  as those given out by the command FILETYPE for identification:

    LEDIS:   A distribution dump created by write instruction »DUMP=…«. Such a dump
	     usually contains all relevant data of the internal model, which is restored
	     by reading it in.

    LEDEF:   A distribution definition file. This new format replaces the old CTTEXT
	     format. It supports all LE-CODE properties and will be further developed to
	     match LE-CODE. Templates can be created with instruction »LEDEF=…« or with
	     command »wlect CREATE LEDEF«.

    LEREF:   A track reference list created by write instruction »REF=…«.

    SHA1REF: A SHA1 reference list created by write instruction »SHA1=…«. Only checksum
	     and track slot are used.

    LE-BIN:  A LE-CODE binary file (e.g. »lecode-PAL.bin«). Usually settings and LPAR are
	     imported. Option IN-LECODE decides if the file is used as template for the
	     specifig region if a binary is created.

    PREFIX:  Replace the internal prefix list by the content of the file.
	     https://ct.wiimm.de/export/prefix is the authoritative source for this.

    LPAR:    A LE-CODE parameter file (e.g. »lpar.txt«).

    BMG and BMGTXT:
	     Any BMG (binary or text). See section »Processing Options« for more details.

    DISTRIB: A distribution file created by write instruction »DISTRIB=…« or by command
	     »wszst DISTRIBUTION«. Such files are imported by ct.wiimm.de to display
	     information about distributions.

    CTTEXT:  A CT-CODE or LE-CODE definition file. The file is scanned by the CT-CODE
	     scanner and then imported.

  All file types that are accepted by option --le-define (BRRES, TEX0, CT-CODE, ...) are
  also possible.


Instructions:

  Instructions are of the form COMMAND=PARAMETER (1 argument) or COMMAND= PARAMETER (2
  arguments). Commands are case-insenitive. The use of keywords without the minus sign
  and unique abbreviations are permitted. Any number of write instructions can be used.
  And these can be mixed with input files at will.

  Most commands, but not all, are write instructions. They write the current contents of
  the internal model to a file. In these cases, PARAMETER is a filename. If a filename is
  a minus sign only, then stdout is used. Existing files are only overwritten if at least
  one of the options --overwrite (short: -o) or --remove-dest (short: -r) is set.

  Create human readable information files:

    NAMES:    Create a human readable reference file with cup info and track names. Only
	      tracks with known name (not empty) are printed. The tracks are ordered by
	      cups.
    XNAMES:   Same as NAMES, but use extended names if available.

    INFO:     Create a human readable reference file with cup info slots and track names.
	      Only tracks with known name (not empty) are printed. The tracks are
	      ordered by cups.
    XINFO:    Same as INFO, but use extended names if available.

    LE-INFO:  Print information about current LE-CODE binaries.

    PREFIX:   Print a machine readable prefix list. https://ct.wiimm.de/export/prefix is
	      the authoritative source for this.

    DEBUG:    Print some statistics for debugging. The current sate is analysed without
	      updating cups.

  Create human and machine readable definition files (usually input files):

    LE-DEF:   Create a LE-CODE definition file of format »#LE-DEF1«, that is usually used
	      to declare tracks for a LE-CODE distributions. This new format replaces the
	      old format created by CTDEF=. It supports all LE-CODE properties and will
	      be further developed to match future LE-CODE.

    CT-DEF:   Create a definition file of format »#CT-CODE«, that is usually used to
	      declare tracks for a CT-CODE or LE-CODE distributions. Only simple layouts
	      are supported, so that the track arrangement can be distorted when
	      importing again. In addition, a file is created that can only be read by
	      the tools since version 2.28a.

  Generate machine-readable files that can also be used as input files:

    SHA1:     Create a SHA1 list file that can be used on Wiimmfi to limit the tracks
	      that can be used in a region. Only tracks with known SHA1 are printed. The
	      output can be used as input file to restore the checksums.
    XSHA1:    Same as SHA1, but use extended names if available.

    DISTRIB:  Create a distribution file like command »wszst DISTRIBUTION« does. However,
	      this variant only supports tracks with a known track slot. The output can
	      be used as input file.
    XDISTRIB: Same as DISTRIB, but use extended names for the track list if available.

    LPAR:     Create a LPAR file. Use --brief (-b) to suppress comments.

    BMG:      Create a BMG binary file. BMG options are recognized. But if the output
	      goes to a terminal, then use BMGTXT insted.
    BMGTXT:   Create a BMG text file. BMG options are recognized.

    REF:      Create a machine readable track reference (type LEREF), that can be used as
	      input file for other commands. One line is printed for each defined track.

    DUMP:     Create a dump of the complete internal model to a distribution file (type
	      LE-DIS). This file can be used as source to restore the internal model.

  Create LE-CODE binary files:

    LECODE:   Create 4 LE-CODE binary files, one for each region. The filename must
	      contain at least one »@« character. The last »@« character is replaced by
	      PAL, USA, JAP and KOR in sequence. The binary data embedded in the SZS
	      tools is usually used as a template. At the moment it is build 32
	      (2022-05-13). Other templates can be used with the IN-LECODE processing
	      option.

    PAL:      Similar to mode LECODE, but only the PAL version is generated. The »@«
	      character has no special meaning here.
    USA:      Like PAL, but the USA version is generated.
    JAP:      Like PAL, but the JAP version is generated.
    KOR:      Like PAL, but the KOR version is generated.

  String functions:

    SUBST:    This command searches for text passages using a regular expression and
	      replaces the found passages with another text.
		Syntax of PARAMETER: STORAGE ',' SEP REGEXP SEP REPLACEMENT SEP OPTIONS

	      STORAGE is a storage indicator explained in section »Processing Options:
	      Storage«. SEP is the very first character behind the comma. It must occur
	      exactly 3 times and separates the 3 parts from each other. REGEXP is the
	      extended regular expression used for searching. If successful, replace that
	      portion matched with REPLACEMENT. The replacement may contain the special
	      string »$0« to refer to that portion of the pattern space which matched,
	      and the special string1 »$1« through »$9« to refer to the corresponding
	      matching sub-expressions in the regexp. The two characters »g« for global
	      (replace all occurrences) and »i« for ignore case are recognized as
	      OPTIONS.

	      Examples: subst=name,+abc+xyz+
			subst=[key],/search ([0-9])/replace $1/gi

    COPY:     This command copies the texts of 1 or more sources to a destination.
		Syntax of PARAMETER: DEST '=' SRC [ '+' SRC ]...

	      DEST and SRC are storage indicators explained in section »Processing
	      Options: Storage«. The sources are joined textually, with CTRL-A (ASCII 1,
	      '\1') inserted as a separator between the sources. Use instruction SUBST to
	      change the separators.

	      Example: copy=[result]=sha1+[size]

  Copy, move or link track files:

    TRACKS: Copy, move or links files following the options --copy-tracks, --move-tracks,
	    --move1-tracks, --link-tracks and --track-dir. PARAMETER is a keyword that
	    specifies the options for execution:

	      NO-LOG: Transfer the tracks without logging.
	      LOG:    Transfer the tracks with logging.
	      TEST:   Log planned actions only and don't touch any file.


Processing Options:

  Processing Options are inserted as comma separated list. A list always begins with a
  plus sign to distinguish it from filenames and instructions. Option names are
  case-insenitive. Unique abbreviations are allowed. Names with a minus sign in their
  name can also be specified without the minus sign.

  Most option names can be preeceded by a minus sign or a slash to negate its meaning.
  Processing Options are only valid for subsequent arguments. This makes it possible to
  use different filters for different input or output files. The keywords are divided
  into several functional groups.

  Managment Options:

    HELP:       Stop execution and print this help.
    RESET:      Reset the filter to its default.

    CUT-STD:    Remove settings and tracks, that are not needed for a standard
		distribution with 32 tracks and 10 battle arenas.
    CUT-CTCODE: Remove settings and tracks, that are not needed for a CT-CODE
		distribution. Arenas are also removed.

  BMG source:

    Define which BMG strings are used as input. Multiple source can be selected. If none
    is selected, then all are used. MKW1 has the lowest priority and LE-CODE the highest.

    MKW1:       Use the first set of orignal names starting from message id 0x2454
		(racing tracks) and 0x2490 (battle arenas).
    MKW2:       Use the second set of orignal names starting from message id 0x24b8
		(racing tracks) and 0x24cc (battle arenas).
    MKW:        Short cut for »+MKW1,MKW2«.
    CT-CODE:    Use the CT-CODE names starting from message id 0x4000.
    LE-CODE:    Use the LE-CODE names starting from message id 0x7000.

  Input filters:

    Define the type of character identifiers accepted as source. Empty character strings
    and character strings that only consist of a minus sign are considered invalid.

    IN-EMPTY:   Empty strings are also considered valid. IEMPTY is a short cut for this
		keyword.
    IN-MINUS:   Strings consisting only of a minus sign are also considered valid. IMINUS
		is a short cut for this keyword.
    IN-ALL:     Short cut for »+IN-EMPTY,IN-MINUS«. IALL is a short cut for this keyword.

  Input operation:

    Define how already existing strings are overwritten.

    OVERWRITE:  Insert all valid BMG strings and overwrite already existing strings. This
		is the default.
    INSERT:     Insert only valid BMG strings, that are empty in the track list.
    REPLACE:    Replace only valid BMG strings that are already defined in the track
		list. Other strings are ignored.

  Storage:

    Define the string type being processed. This is used for BMG input and output files
    only.

    IDENT:      Define a new identfication string. If it is a valid SHA1 or DB64 string,
		then update SHA1 too. If reading, then get the identfication string.
    SHA1:       Same as IDENT if writing. If reading, then get the SHA1 string.
    FILE:       Define a new or get the filename.
    NAME:       Define a new or get the standard name.
    XNAME:      Define a new or get the extended name.
    X2NAME:     Define a new standard and a new extended name. If reading, then get the
		extended name, if valid, or the standard name as fallback.

    [key]:      For each track there is a set of character strings at the user's
		disposal. The character strings are addressed via KEY. The KEY itself can
		consist of any character and are case-sensitive. This type of square
		bracket option can only be used directly after the leading plus sign.
		More processing options may follow.

  Output filters:

    Select tracks for the output by their features.

    VERSUS|VS:  Select versus tracks only.
    BATTLE|BT:  Select battle tracks (arenas) only. If neither or both of VERSUS and
		BATTLE set, both types are selected.

    CUSTOM:     Select custom tracks. An original track in the wrong slot will also be
		considered CUSTOM.
    ORIGINAL:   Select original tracks at correct track slot only. Original tracks are
		detected by their SHA1. If neither or both of CUSTOM and ORIGINAL set,
		both types are selected.

    NO-D:       This option suppresses information about so-called '_d' files. This
		affects 2 sub-commands: With the REF sub-command, the data fields
		relating to '_d' files remain empty, and with SHA1, no lines are output
		for '_d' tracks.

  Empty output strings:

    For sub-commands (X)NAMES and (X)INFO, only tracks that have a valid name are output.
    Empty character strings and character strings that only consist of a minus sign are
    considered invalid.

    OUT-EMPTY:  Empty strings are also considered valid. OEMPTY is a short cut for this
		keyword.
    OUT-MINUS:  Strings consisting only of a minus sign are also considered valid. OMINUS
		is a short cut for this keyword.
    OUT-ALL:    Short cut for »+OUT-EMPTY,OUT-MINUS«. OALL is a short cut for this
		keyword.


  More options:

    IN-LECODE:  When reading a LE-CODE binaray file (e.g. »lecode-PAL.bin«), only the
		settings are imported, but not the binary itself. So if writing a LE-CODE
		binary then the region dependent built-in binary is used. If this option
		set, then binaries are imported to override the built-in versions. One
		reason to activate this option is to use more recent LE-CODE versions.
		ILECODE is a short cut for this keyword.
    /IN-LECODE: Switch option IN-LECODE off.

    BRIEF:      When creating text files, detailed descriptions are suppressed. The
		default is based on the global options --no-header and --brief.
    /BRIEF:     Switch option BRIEF off.


Examples:

  ....


Built-in LE-CODE binaries:

  PAL v4, build 32 (2022-05-13 21:36:11 UTC), 71584 bytes
  USA v4, build 32 (2022-05-13 21:36:16 UTC), 71584 bytes
  JAP v4, build 32 (2022-05-13 21:36:21 UTC), 71584 bytes
  KOR v4, build 32 (2022-05-13 21:36:25 UTC), 71584 bytes