Melody - CSV File Format (MCSV)
======================================

Rev: 1.0
Author: KF
Date: 14.10.2005

1. Introduction:
--------------------------------------------------------------------------------------------------------

The Melody-CSV (MCSV) file format proposed in this document is another proprietrary 
symbolic format for the description of (currently) monophonic melodies. It is based on the 
well-known CSV (Comma Separated Values) file format. The advantages of this approach
is the easy readibility for certain standard software like spreadsheets and (e.g. EXCEL), 
statistics applications (e.g. SPSS) or databases (e.g. ACCESS). Another advantage is due to the column 
oriented approach with means easy extensibility. In this draft the main file layout,
mandatory and some optional header and data fields are described.

The basic idea is that of a list of events tagged with time points resulting in a form of time series.
Any data in the data section can be basically of three semantic types:
"time", "duration" or "event". Time data relates to points in absolute time, duration data relates to 
time spans (e.g. differentiated time), and event data described any kind of event, coded in some form,
though mostly describing pitch events. 

1.1 Notation:
--------------------------------------------------------------------------------------------------------

Throughout this text we use some specific notations inspired by some kind of Backus-Naur-Form, literals 
(i.e. strings and terminal symbols mare enclosed in quotation marks, The pipe symbol "|" indicates alternatives, 
brackets grouping. Options are indicate with square brackets. We use the following data types, which are indicated 
with angle brackets:

Name   Meaning
=============================================
float      floating point value, either in German or English notation (c.f. below);
int        integer value
uint       integer value >= 0
pint       integer value > 0
string     string of printable characters
pow2       power of two, i.e. 1,2,4,8...
major      "A"|"B"|"C"|"D"|"E"|"F"|"G"
minor      "a"|"b"|"c"|"d"|"e"|"f"|"g"
hepstep    "1"|"2"|"3"|"4"|"5"|"6"|"7"
byte       integer value 0-255
bool       "true"|"false"
format     "csv"|"midi"|"esac"|"kern"|"audio"|<string>
guid       unique guid
composer   "unknown"|"trad."|"app"|<string>
tempoind   ("ms"|"bpm")":"<float>
sigind     <pint>"/"<pow2>
keyind     <major>|<minor>|<major><hepstep>

To differentiate between literals and numeric variable we use curly brackets around numeric variables. So
{n}/2^{m} can be any string like 2/4, 7/16 etc. For numeric variables in the data section the (zero-based) index 
notation with square brackets means using the n+1-th data line. E.g. {onset[n]} means the value of onset in 
line {n+1} of the data section.

2. Main File Layout
--------------------------------------------------------------------------------------------------------

The main layout of an MCSV file is 
      <file header><data header><data>[<blankline><data header><data>[... ]] 
whereby the header lines are identified with a starting '#' sign. A line beginning 
with '##' should be considered as a commentary and can occur every where in the file.

The header lines define global and usually necessary data, like overall time 
signature, tempo, key information etc. The data section of the document MUST 
contain a data header line defining the meaning of the subsequnet data lines.
The data can be written in two different modes: comma or semicolon mode (CM, SM). 
In the first case each data field is separated by a comma, in the second case with 
a semicolon. Additionally floating point numbers can be represented in two different 
formats: English or German float format (EFF, GFF). In the first case floating point number 
use the usual dot ('.') as decimal separators, in the second case a comma (',') 
is used. It is clear, that CM and GFF cannot be used together. 

Table of combinable modes and formats
=====================================
   CM EFF
   SM EFF
   SM GFF

2.1. Header fields
--------------------------------------------------------------------------------------------------------

The global header lines must obey the syntax
   #<keyword>=<value>

The following table shows pre-defined keywords and allowed values:

Keyword           Values            State
=================================================================
"timeunit"        "ms"|"s"	        mandatory
"separator"       ","|";"           mandatory
"floatformat"     "."|","           mandatory
"polyphonic"      <bool>            optional
"key"             <keyind>          optional
"signature"       <sigind>          optional
"tempo"           <keyind>          optional
"division"        <pint>            optional
"tatum"           <pint>            optional
"sourcefile"      "self"|<string>   optional
"sourceformat"    <format>          optional
"generator"       <string>          optional
"name"            <string>          optional
"id"              <guid>            optional
"composer"        <composer>        optional
"ref"             <string>          optional
"addinfo"         <string>          optional

2.1.1 "timeunit"
The field "timeunit" indicates whether the time data given should be interpreted as seconds (s)or milliseconds. 
This refers to "onset" and "durs" and any custom defined time data fields.

2.1.2 "separator"
The field "separator" indicates whether the data is separated by commas or semicolons.

2.1.3 "floatformat"
The value of "floatformat" indicates whether floating points data is represented in english or german 
mode.

2.1.4 "polyphonic"
This field MUST set to "false".  In further revisions of the MCSV format enhancements to 
polyphonic data might be included.

2.1.5 "key"
This field indicates the overall key of the piece. Upper case letter are used to indicate major, lower case letter 
to indicate minor modes. Alternatively the syntax <A-G>{n}, can be used, where "n" indicates the heptatonic starting 
step of the mode. E.g. A minor (= A aeolic) mode is equivalent to C6, D dorian to C2 etc. 

2.1.6 "signature" 
The field "signature" gives the overall time signature of the piece. n, m can be any positive integer.

2.1.7 "tempo" 
The "tempo" states extra tempo information, either as beat per minutes (bpm) or as milliseconds (ms) per beat. 
E.g. bpm:120 is equivalent to ms:500. The transformation formulas are 
	ms = 60000/bpm
	bpm = 60000/ms
	
2.1.8 "division"
Like in MIDI files, "division" gives the number of ticks per beat.

2.1.9 "tatum"
Like ticks, "tatum" indicates a musical time unit per beat. The meaning E.g. a tatum value of 2 along with a signature 
of 4/4 indicates two eights per beat as smallest unit used for this melody.

2.1.10 "sourcefile"
Very often it will be the case that MCSV files are generated through a conversion tool from another file. If this is
the case the conversion tool can write the original filename in this field. Otherwise the value "self" indicates an 
original MCSV file.

2.1.11 "sourceformat"
If the MCSV file is generated from another file in a different format, then this field should indicate the original format
of the source file

2.1.12 "generator"
The identification (e.g., name and version) of the application that generated the file.

2.1.13 "name"
The "name" field should contain the name or title of the song presented in the MSCV file

2.1.14 "id"
For book keeping, identifiaction and database purposes it could be useful to imprint each MSCV file with an unique ID, 
given as a GUID.

2.1.15 "composer"
The name of the composer of this melody, if known. For folk melodies you should use the keyword "trad.", for computer 
generated melodies use "app", if there is a composer, but the name is not available use "unknown".

2.1.16 "ref"
Here a reference to the melody contained in the file can be given, might be an URL or a identification of a CD that contains this melody or any other reference.

2.1.4 "addinfo"
This string valued field can contain additional information of any kind for the given file, which can not be covered 
otherwise in the header.

 
2.2. Data Header and data fields
--------------------------------------------------------------------------------------------------------

The data header is absolute mandatory, as well as a certain subset of predefined data header fields. The data header 
line itself is a comma (semicolon) separated  list of header fields. The keywords describing 
the data in the corresponding column MUST NOT exceed 8 characters (is is due to a limitation in SPSS).
The following table gives an overview of the predefined keywords and the appropiate data types for the data lines.


Keyword        Datatype             State          Semantic type
=================================================================
"onset"        <float>              mandatory      time
"onsetics"     <uint>               optional       time
"bar"          <uint>               optional *)    time
"beat"         <uint>               optional *)    time
"ticks"        <uint>               optional *)    time
"pitch"        <byte>               mandatory      event
"pitchf"       <float>              optional       event
"durs"         <float>              mandatory      time
"durtic"       <uint>               optional       time
"dur16"        <int>                optional       time
"tempo"        <tempoind>|"*"       optional       event
"division"     <pint>|"*"           optional       event
"signature"    <sigind>|"*"         optional       event

*) If any of this fields is given, all three must be present.

2.2.1 Explanation of data fields
-----------------------------------

2.2.1.1 "onset"
The "onset" value is given either in seconds or milliseconds according to the header definition. The meaning is the start 
time of an event (usually pitch event). The MSCV data lines MUST be increasibsly ordered according to "onsets" 
(or "onsetics") values. I.e. for each pair of lines {n} and {m} the following relations MUST hold
      {onset[n]}     > {onset[m]}   for {n}>{m} 
	   {onsetic[n]}   > {onsetic[m]} for {n}>{m}
	   
A MCSV file is called time-normalized if {onset[0]} = 0.

2.2.1.2 "onsetic" 
The "onsetic" value holds essentially the same information as the "onset" value the only difference being that "onsetic" 
is an integer value. The following relation holds:
		{onset[n]} = {onsetic[n]-onset[0]}/{division} * {tempo} + {onset[0]} ({tempo} given in milliseconds)
or
		{onset[n]} = {onsetic[n]-onset[0]}/{division} * 60000/{tempo} + {onset[0]} ({tempo} given as beats per minute)

2.2.1.4 "bar", "beat", "ticks"
This three values express the onset information of an event in cyclic integer time. This means, that time is organized 
in beats the timespan between two beats being constant. Each beat is divided in {division} ticks, each ticks last 
{tempo}/{division} milliseconds (if tempo is given in milliseconds per beat). {n} beats form a bar, accordingly each 
bar contains {n}*{division} ticks, where {n} is given by the signature. After {n} beats the bar number is increased by 
one. By convention beats are counted from 1 to {n}, ticks from 0 to {division}-1. 

2.2.1.4 "pitch"
The "pitch" data field is an integer number between 0 and 255 (1 byte). The numbers between 0 and 127 accord to the 
pitches of the equally tempered scale as defined in the MIDI standard, where 60 corresponds to C' (middle C).
The values from above 128 are reserved for some special meaning, indicating (pseudo-)events other than pitch events.
Currently the following non-pitch events are defined

Pitch value		Meaning
==================================
  128     Null event, silence
  129     Continue event 
  130     Harmony change  
  255     Undefined event

2.2.1.5 "pitchf"
The "pitchf" data field indicates pitch given in absolute frequency. The transformation between the two values is 
given by the formula:
      {pitch} = INT(12 * log2(({pitchf}/440)) + 69
and
	  {pitchf} = 440 * 2^({pitch}-69)/12) (Hz)
	  
2.2.1.6 "durs"
This field is the duration of the event in the defined time unit (s or ms). There is a constraint to this value due 
to the time series structure of the event list:
		{durs[n]}<= {onset[n+1]} - {onset[n]}

2.2.1.7 "durtic"
This field is about the same inforamtion as "durs", only that the duration is given in ticks. One of these fields
is mandatory. The conversion formula is:
		{durs} = {durtic}/{division} * {tempo} ({tempo} given in milliseconds)
or
		{durs} = {durtic}/{division} * 60000/{tempo} ({tempo} given as beats per minute)
Accordingly the constraint to "durtic" is as follows:
		{durtics[n]}<= {onsetics[n+1]} - {onsetics[n]}

2.2.1.8 "dur16"
Like "durs" and "durtic" this field indicates the duration of an event, but given in smallest musical time units 
as defined in the header field "tatum". If no "tatum" is defined, a default value of 4 should be used. Negative 
values of "dur16" indicated fractions of the tatum (and so undermining somehow the notion of smallest time unit...).
A negative value of -{d} means a duration of {tatum}/{d+1}, e.g {dur16} = -2 means a duration of {tatum}/3. 

2.2.1.9 "tempo"
The "tempo" field indicates a tempo change. This is mainly important for the relation of the data fields of time 
and duration type. A tempo event occuring along with a pitch event instantly changes the tempo frame for this event.
A value "*" means that the tempo is not changed. A tempo event given in the first line of the data section 
overrides instantly the tempo information given in the header fields.

2.2.1.10 "division"
Like the tempo event the "division" event changes instantly the basic division of a beat. A value of "*" means no change.

2.2.1.11 "signature"
A signature event changes instantly the signature of the presented melody. A value of "*" means no change of signature.


   