OpenMPT Wiki - User contributions [en]

Development: Formats/IT

2015-05-10T17:44:22Z

Uyjulian: /* OpenSPC conversions */

General information concerning the IT file format. Any variable names in this document refer to the original variable names from [http://schismtracker.org/wiki/ITTECH.TXT ITTECH.TXT]. Variable names and hexadecimal numbers (in little-endian formats) are indicated by a <tt>monospace font</tt>.

Note: Impulse tracker is now open source! You can grab the [https://bitbucket.org/jthlim/impulsetracker source code] at Bitbucket.

== Tracker IDs ==

The IT header contains a field, <tt>Cwt/v</tt>, which is used to identify the application that was used to create the file.
The following custom tracker IDs (read as little-endian hexadecimal numbers) are known to be found in the <tt>Cwt/v</tt> field:

* <tt>0xyy</tt> - Impulse Tracker x.yy - Many other trackers, like ModPlug Tracker, disguise as various versions of Impulse Tracker, so this is <strong>not</strong> a reliable way to tell if a file was really made with Impulse Tracker. See [[#Detecting other trackers|below]] for detecting such trackers.
* <tt>1xyy</tt> - Schism Tracker x.yy (up to Schism Tracker v0.50, later versions encode a timestamp in the version number - see Schism Tracker’s <tt>[http://schismtracker.org/hg/file/tip/schism/version.c version.c]</tt>)
* <tt>4xyy</tt> - pyIT x.yy
* <tt>5xyy</tt> - OpenMPT x.yy
* <tt>6xyy</tt> - BeRoTracker x.yy
* <tt>7xyz</tt> - ITMCK x.y.z (N.B. the user can override the value of the <tt>Cwt/v</tt> header by the <tt>-w</tt> command-line switch or the <tt>#TRACKER-VERSION</tt> command in the input file)
* <tt>DAEB</tt> - spc2it (after commit d3d1b524032f668be8ee20d25339e1c635fb9bcf)

Are you a tracker developer and you tracker can save IT files? Please add your Tracker ID to this list!

== Detecting other trackers ==

Some trackers disguise as Impulse Tracker 2, but there are ways to uncover this diguise:

=== BeRoTracker ===

Earlier versions of BeRoTracker, which did not use the tracker ID described above, can be identified by the four byte <tt>MODU</tt> magic which can be found after the Edit History / MIDI Macro block.

=== ChibiTracker ===

<tt>Cwt/v</tt> and <tt>Cmwt</tt> are both <tt>0214</tt>, <tt>reserved</tt> reads <tt>CHBI</tt>.

Further interesting criteria: Sample data is stored directly after header. All sample and instrument filenames read <tt>-DEPRECATED-</tt> (obviously this should not be used as a detection criteria, as files re-saved with other trackers will still have these filenames). ChibiTracker also uses a line feed character (<tt>0A</tt> or <tt>\n</tt>) instead of a carriage return (<tt>0D</tt> or <tt>\r</tt>) in the song message.

=== ModPlug Tracker ===

* '''ModPlug Tracker 1.00a5''': <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0200</tt>, <tt>reserved</tt> is <tt>0</tt>. Instruments are 560 bytes apart.
* '''ModPlug Tracker 1.00b3.3 - 1.07''': <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0202</tt>, <tt>reserved</tt> is <tt>0</tt>. Instruments are 557 bytes apart.
* Newer versions of '''ModPlug Tracker''', '''OpenMPT 1.17 in compatible mode''': <tt>Cwt/v</tt> is <tt>0217</tt>, <tt>Cmwt</tt> is <tt>0200</tt>, <tt>reserved</tt> is <tt>0</tt>. If there are any instruments in the file, the instrument <tt>TrkVers</tt> field is <tt>0211</tt> for ModPlug Tracker and <tt>0220</tt> for OpenMPT. ModPlug Tracker writes <tt>FF</tt> for unused channels in the panning map, OpenMPT never does this.

=== OpenMPT ===

OpenMPT 1.17 wrote the value <tt>0888</tt> in both the <tt>Cwt/v</tt> and <tt>Cmwt</tt> fields.

Later versions of OpenMPT use the tracker ID provided above. In that case, the <tt>reserved</tt> field reads <tt>OMPT</tt> if the file was saved in normal mode, and is <tt>0</tt> if it was saved in compatible mode.

=== OpenSPC conversions ===

OpenSPC is a tool to convert SNES music from the SPC format to the IT format. If all the following conditions are true, you can be fairly sure that a module was written with OpenSPC:
* <tt>Cwt/v</tt> is <tt>0214</tt>.
* <tt>Cmwt</tt> is <tt>0200</tt>.
* <tt>Flags</tt> is <tt>09</tt> (Linear Slides, Stereo Playback).
* <tt>Special</tt> is <tt>0</tt>.
* Both pattern highlight values are <tt>00</tt>.
* No instruments are used.
* <tt>PatNum</tt> + 1 == <tt>OrdNum</tt>.
* Global volume is 128 decimal.
* Master volume is 100 decimal.
* Initial Speed is 1.
* Pan Separation is 128 (decimal).
* Pitch wheel depth is 0.
* There is no song message, i.e. message length and offset are 0.
* <tt>reserved</tt> is 0.

Earlier versions of spc2it (before commit d3d1b524032f668be8ee20d25339e1c635fb9bcf) will be mis-identified as OpenSPC, as the tool closely follows the way its predecessor writes out IT files.

=== UNMO3 ===

UNMO3 is a tool to convert an MO3-compressed module file back into its original format.

In IT files created with UNMO3, <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0214</tt>, <tt>reserved</tt> is <tt>0</tt>. The <tt>Special</tt> bit 2 (embed row highlights) is never set.

'''Note''': UNMO3 older than version 2.4.0.1 does not set the edit history flag, but writes two edit history length bytes (zeroes) in the file anyway. Keep this in mind, as MIDI Macros will not load properly if you don’t. A fairly reliable (but not perfect) way to detect such malformed IT files would be to try to read the edit history length if all of the above conditions are fulfilled '''and''' the edit history flag is not set. For these old versions of UNMO3, you should read two zero bytes. If that is the case, it means that the first MIDI macro would be empty. This cannot happen in our case because UNMO3 writes out default values for the nine global macros, and the first default value is “FF” (MIDI Start), so it is clear that the two zero bytes belong to the (empty) edit history.

Later version of UNMO3 set the correct header flag for the edit history and write the two zero length bytes.

== Special Header Flags ==

The following "Special" flags are not documented:

* Bit 1 (<tt>02</tt>): If set, edit history information (see [[#Edit History|below]]) is present.
* Bit 2 (<tt>04</tt>): If set, the row highlights (at offset <tt>1E</tt> - <tt>1F</tt>) should be read. If not set, they should be ignored.

== Edit History ==

It is little known that the latest versions of Impulse Tracker keep track of when an IT file is opened and how long it is left open.
The edit history can be found directly after the IT header and before the possibly present MIDI macro configuration.

A 16-Bit integer <tt>num</tt> tells how many edit history blocks are present. Every block is 8 bytes long, so the 16-bit integer is followed by <tt>8 * num</tt> bytes of edit history information.

Each history block consists of a 16-Bit FAT date, a 16-Bit FAT time and a 32-Bit DOS timer, which contains the time how long the file was open in the editor, in 1/18.2th seconds.
Information on how to decode the FAT date and time stamps can be found on the [http://msdn.microsoft.com/en-us/library/windows/desktop/ms724247%28v=vs.85%29.aspx MSDN].

Note: Many programs don’t write an edit history, but they set the edit history flag in the header and write the two zero bytes for the history length. Many of them also ignore the edit history flag when loading files and always read the two <tt>num</tt> num bytes (ignoring the special flag), so if you want to write out an IT file that does not have an edit history and do not mind about the two extra bytes, the safest (most compatible) way is to set the edit history flag and write two zero bytes for <tt>num</tt>.

Also note that none of this is really relevant if you are sure that you never write out the MIDI macro configuration or ModPlug extensions, since these are the only two things that could normally follow the edit history. So if you have no MIDI macros and no ModPlug extensions, it is always safe to not set the special flag and not write the two extra bytes.

== Proposed way to store compressed stereo samples ==

Since Impulse Tracker itself does not support stereo samples, a common way of how to decode compressed stereo samples had to be found - After all, it is possible to just treat them as one continuous stream (not stopping the encoding / decoding process after the end of the left channel), or to treat them as two separate, consecutive streams. GreaseMonkey has decided to follow [http://www.un4seen.com/forum/?topic=12448.msg86479#msg86479 XMPlay's way] of decoding compressed stereo samples by treating them as two streams of the same length, so OpenMPT follows this example. If you intend to add IT sample compression support in your application, you should do the same to ensure compatibility across as many applications as possible.

[[Category:Development|Formats/IT]]
[[Category:IT Format|Format Specification]]

Development: Formats/IT

2015-03-18T00:54:18Z

Uyjulian:

General information concerning the IT file format. Any variable names in this document refer to the original variable names from [http://schismtracker.org/wiki/ITTECH.TXT ITTECH.TXT]. Variable names and hexadecimal numbers (in little-endian formats) are indicated by a <tt>monospace font</tt>.

Note: Impulse tracker is now open source! You can grab the [https://bitbucket.org/jthlim/impulsetracker source code] at Bitbucket.

== Tracker IDs ==

The IT header contains a field, <tt>Cwt/v</tt>, which is used to identify the application that was used to create the file.
The following custom tracker IDs (read as little-endian hexadecimal numbers) are known to be found in the <tt>Cwt/v</tt> field:

* <tt>0xyy</tt> - Impulse Tracker x.yy - Many other trackers, like ModPlug Tracker, disguise as various versions of Impulse Tracker, so this is <strong>not</strong> a reliable way to tell if a file was really made with Impulse Tracker. See [[#Detecting other trackers|below]] for detecting such trackers.
* <tt>1xyy</tt> - Schism Tracker x.yy (up to Schism Tracker v0.50, later versions encode a timestamp in the version number - see Schism Tracker’s <tt>[http://schismtracker.org/hg/file/tip/schism/version.c version.c]</tt>)
* <tt>4xyy</tt> - pyIT x.yy
* <tt>5xyy</tt> - OpenMPT x.yy
* <tt>6xyy</tt> - BeRoTracker x.yy
* <tt>7xyz</tt> - ITMCK x.y.z (N.B. the user can override the value of the <tt>Cwt/v</tt> header by the <tt>-w</tt> command-line switch or the <tt>#TRACKER-VERSION</tt> command in the input file)
* <tt>DAEB</tt> - spc2it (after commit d3d1b524032f668be8ee20d25339e1c635fb9bcf)

Are you a tracker developer and you tracker can save IT files? Please add your Tracker ID to this list!

== Detecting other trackers ==

Some trackers disguise as Impulse Tracker 2, but there are ways to uncover this diguise:

=== BeRoTracker ===

Earlier versions of BeRoTracker, which did not use the tracker ID described above, can be identified by the four byte <tt>MODU</tt> magic which can be found after the Edit History / MIDI Macro block.

=== ChibiTracker ===

<tt>Cwt/v</tt> and <tt>Cmwt</tt> are both <tt>0214</tt>, <tt>reserved</tt> reads <tt>CHBI</tt>.

Further interesting criteria: Sample data is stored directly after header. All sample and instrument filenames read <tt>-DEPRECATED-</tt> (obviously this should not be used as a detection criteria, as files re-saved with other trackers will still have these filenames). ChibiTracker also uses a line feed character (<tt>0A</tt> or <tt>\n</tt>) instead of a carriage return (<tt>0D</tt> or <tt>\r</tt>) in the song message.

=== ModPlug Tracker ===

* '''ModPlug Tracker 1.00a5''': <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0200</tt>, <tt>reserved</tt> is <tt>0</tt>. Instruments are 560 bytes apart.
* '''ModPlug Tracker 1.00b3.3 - 1.07''': <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0202</tt>, <tt>reserved</tt> is <tt>0</tt>. Instruments are 557 bytes apart.
* Newer versions of '''ModPlug Tracker''', '''OpenMPT 1.17 in compatible mode''': <tt>Cwt/v</tt> is <tt>0217</tt>, <tt>Cmwt</tt> is <tt>0200</tt>, <tt>reserved</tt> is <tt>0</tt>. If there are any instruments in the file, the instrument <tt>TrkVers</tt> field is <tt>0211</tt> for ModPlug Tracker and <tt>0220</tt> for OpenMPT. ModPlug Tracker writes <tt>FF</tt> for unused channels in the panning map, OpenMPT never does this.

=== OpenMPT ===

OpenMPT 1.17 wrote the value <tt>0888</tt> in both the <tt>Cwt/v</tt> and <tt>Cmwt</tt> fields.

Later versions of OpenMPT use the tracker ID provided above. In that case, the <tt>reserved</tt> field reads <tt>OMPT</tt> if the file was saved in normal mode, and is <tt>0</tt> if it was saved in compatible mode.

=== OpenSPC conversions ===

OpenSPC is a tool to convert SNES music from the SPC format to the IT format. If all the following conditions are true, you can be fairly sure that a module was written with OpenSPC:
* <tt>Cwt/v</tt> is <tt>0214</tt>.
* <tt>Cmwt</tt> is <tt>0200</tt>.
* <tt>Flags</tt> is <tt>09</tt> (Linear Slides, Stereo Playback).
* <tt>Special</tt> is <tt>0</tt>.
* Both pattern highlight values are <tt>00</tt>.
* No instruments are used.
* <tt>PatNum</tt> + 1 == <tt>OrdNum</tt>.
* Global volume is 128 decimal.
* Master volume is 100 decimal.
* Initial Speed is 1.
* Pan Separation is 128 (decimal).
* Pitch wheel depth is 0.
* There is no song message, i.e. message length and offset are 0.
* <tt>reserved</tt> is 0.

=== UNMO3 ===

UNMO3 is a tool to convert an MO3-compressed module file back into its original format.

In IT files created with UNMO3, <tt>Cwt/v</tt> is <tt>0214</tt>, <tt>Cmwt</tt> is <tt>0214</tt>, <tt>reserved</tt> is <tt>0</tt>. The <tt>Special</tt> bit 2 (embed row highlights) is never set.

'''Note''': UNMO3 older than version 2.4.0.1 does not set the edit history flag, but writes two edit history length bytes (zeroes) in the file anyway. Keep this in mind, as MIDI Macros will not load properly if you don’t. A fairly reliable (but not perfect) way to detect such malformed IT files would be to try to read the edit history length if all of the above conditions are fulfilled '''and''' the edit history flag is not set. For these old versions of UNMO3, you should read two zero bytes. If that is the case, it means that the first MIDI macro would be empty. This cannot happen in our case because UNMO3 writes out default values for the nine global macros, and the first default value is “FF” (MIDI Start), so it is clear that the two zero bytes belong to the (empty) edit history.

Later version of UNMO3 set the correct header flag for the edit history and write the two zero length bytes.

== Special Header Flags ==

The following "Special" flags are not documented:

* Bit 1 (<tt>02</tt>): If set, edit history information (see [[#Edit History|below]]) is present.
* Bit 2 (<tt>04</tt>): If set, the row highlights (at offset <tt>1E</tt> - <tt>1F</tt>) should be read. If not set, they should be ignored.

== Edit History ==

It is little known that the latest versions of Impulse Tracker keep track of when an IT file is opened and how long it is left open.
The edit history can be found directly after the IT header and before the possibly present MIDI macro configuration.

A 16-Bit integer <tt>num</tt> tells how many edit history blocks are present. Every block is 8 bytes long, so the 16-bit integer is followed by <tt>8 * num</tt> bytes of edit history information.

Each history block consists of a 16-Bit FAT date, a 16-Bit FAT time and a 32-Bit DOS timer, which contains the time how long the file was open in the editor, in 1/18.2th seconds.
Information on how to decode the FAT date and time stamps can be found on the [http://msdn.microsoft.com/en-us/library/windows/desktop/ms724247%28v=vs.85%29.aspx MSDN].

Note: Many programs don’t write an edit history, but they set the edit history flag in the header and write the two zero bytes for the history length. Many of them also ignore the edit history flag when loading files and always read the two <tt>num</tt> num bytes (ignoring the special flag), so if you want to write out an IT file that does not have an edit history and do not mind about the two extra bytes, the safest (most compatible) way is to set the edit history flag and write two zero bytes for <tt>num</tt>.

== Proposed way to store compressed stereo samples ==

Since Impulse Tracker itself does not support stereo samples, a common way of how to decode compressed stereo samples had to be found - After all, it is possible to just treat them as one continuous stream (not stopping the encoding / decoding process after the end of the left channel), or to treat them as two separate, consecutive streams. GreaseMonkey has decided to follow [http://www.un4seen.com/forum/?topic=12448.msg86479#msg86479 XMPlay's way] of decoding compressed stereo samples by treating them as two streams of the same length, so OpenMPT follows this example. If you intend to add IT sample compression support in your application, you should do the same to ensure compatibility across as many applications as possible.

[[Category:Development|Formats/IT]]
[[Category:IT Format|Format Specification]]