Xylophone gag project - Revision history

Seth at 02:53, 27 April 2010

2010-04-27T02:53:21Z

Seth at 02:51, 27 April 2010

2010-04-27T02:51:55Z

Seth at 02:47, 27 April 2010

2010-04-27T02:47:53Z

Seth at 02:39, 27 April 2010

2010-04-27T02:39:31Z

Seth at 02:31, 27 April 2010

2010-04-27T02:31:29Z

Seth at 02:18, 27 April 2010

2010-04-27T02:18:28Z

Seth at 02:16, 27 April 2010

2010-04-27T02:16:24Z

Seth at 02:10, 27 April 2010

2010-04-27T02:10:16Z

Seth at 02:06, 27 April 2010

2010-04-27T02:06:15Z

Seth at 01:49, 27 April 2010

2010-04-27T01:49:34Z

@@ Line 101: / Line 101: @@
 |}
 -->
-====Yosemite Sam====
+====Yosemite Sam frustrated on piano====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"
+|
-Yosemite Sam's correct, lethal, attempt at the song
+Yosemite Sam's correct, lethal, attempt at the song, cut off just before explosion.
 #[http://enfascination.com/forever/xylophonegag/FinalYosemiteOriginal.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalYosemiteConverted.wav Recognized representation]
@@ Line 114: / Line 114: @@
 ===bad identifications===
-====Bugs Bunny playing====
+====Bugs Bunny playing piano====
 Deferring to the holy law of triples, Bugs Bunny attempts the tune twice before Sam gets frustrated and gets it right (wrong) the third time.  My attempt at catching Bugs Bunny's mistakes failed due to the limits of the simple recognizer that I built on top of, evident in the screenshots.  This may be due to recording quality, or maybe the fact that it is played on piano.  I am tempted to rule that out though, because I recognized Yosemite Sam fine, and he played on the same piano.
 {| cellpadding="2" style="border: 0px;"
@@ Line 136: / Line 136: @@
 |}
-====guitar playing====
+====guitar====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"

@@ Line 36: / Line 36: @@
 In the images below, the horizontal axis is time (in frames of 2/100s of a second) and the vertical axis is recognized midi pitch (which you will see is jumpy).  These are all graphs of a same tune.  It rises in pitch.  lines on the bottom edge of a plot represent silence.  The canonical mistake, that bugs makes, is that the very last note should be C, but ends up too high or low.
 ===caught mistakes===
-====canonical mistake (on penny whistle)====
+====canonical player error (on penny whistle)====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"
@@ Line 46: / Line 46: @@
 [[Image:FinalErrorPos1112Converted.jpg|right|400px|thumb| canonical mistake (on penny whistle) ]]
 |}
-====canonical mistake with detection error====
+====canonical player error with detection mistake====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"
@@ Line 56: / Line 56: @@
 [[Image:FinalErrorPos41112Converted.jpg|right|400 px|thumb|note detection error at time 200. Representation is flat.]]
 |}
-====other mistake (on recorder)====
+====other error and mistake (on recorder)====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"
@@ Line 69: / Line 69: @@
 ===good versions (no mistakes played and no mistakes found)===
 {| cellpadding="2" style="border: 0px;"
 |- border="0"
@@ Line 100: / Line 101: @@
 |}
 -->
-----
+====Yosemite Sam====
 {| cellpadding="2" style="border: 0px;"
 |- border="0"

@@ Line 4: / Line 4: @@
 The goal was to apply machine learning techniques to digital audio to identify a player's mistakes playing a known melody.  This is a course project[http://www.music.informatics.indiana.edu/courses/I547/] in only about 500 lines of code, so there are severe restrictions on its general utility.  It works on monophonic, single instrument melodies, and works better if the melody is short and played on a wind instrument.  It does not identify timing errors and wasn't tested on skipped notes, only wrong notes.  To keep close roots to the gag, I only benchmarked the code on the classic tune for the Irish poem, "[http://en.wikipedia.org/wiki/Believe_Me,_if_All_Those_Endearing_Young_Charms Believe me, if all those endearing young charms]."[http://www.contemplator.com/ireland/believe.html]
-However, I do demonstrate the robustness of the code to identify mistakes across the melody, as played in different keys, at different tempos, and across a few instruments.  For my simple test examples, and given the limits of the HMM that I built the code on, this system avoids both false positives and negatives (thinking there is a mistake when there isn't and thinking there is not a mistake when there is).
+However, I do demonstrate the robustness of the code to identify mistakes across the melody, as played in different keys, at different tempos, and across a few instruments.  Also different performers (thank you Nathaniel, Travis, Bugs Bunny and Yosemite Sam).  For my simple test examples, and given the limits of the HMM that I built the code on, this system successfully identifies mistakes, avoiding both false positives and negatives (thinking there is a mistake when there isn't and thinking there is not a mistake when there is).
-I modified a simple HMM implementation[http://www.music.informatics.indiana.edu/courses/I547/hw6_sln.r][http://www.music.informatics.indiana.edu/courses/I547/] written in [http://www.r-project.org R] to have rudimentary score following.  I then introduced an obnoxious screech into the audio to indicate identification of each mistake.
+To implement this, I modified a simple HMM [http://www.music.informatics.indiana.edu/courses/I547/hw6_sln.r][http://www.music.informatics.indiana.edu/courses/I547/] written in [http://www.r-project.org R] to add rudimentary score following.  I then introduced an obnoxious screech into the audio to indicate identification of each mistake.
 ==the tune==
@@ Line 112: / Line 112: @@
 |}
-===bad identifications (Bugs Bunny playing)===
+===bad identifications===
 Deferring to the holy law of triples, Bugs Bunny attempts the tune twice before Sam gets frustrated and gets it right (wrong) the third time.  My attempt at catching Bugs Bunny's mistakes failed due to the limits of the simple recognizer that I built on top of, evident in the screenshots.  This may be due to recording quality, or maybe the fact that it is played on piano.  I am tempted to rule that out though, because I recognized Yosemite Sam fine, and he played on the same piano.
 {| cellpadding="2" style="border: 0px;"
@@ Line 133: / Line 134: @@
 [[Image:FinalBugsBunny01Converted.jpg|right|400px]]
 |}
-----
 {| cellpadding="2" style="border: 0px;"
 |- border="0"

@@ Line 132: / Line 132: @@
+|
 [[Image:FinalBugsBunny01Converted.jpg|right|400px]]
 |}

@@ Line 2: / Line 2: @@
 '''Youtube broadcast, starting at 5:45, until 6:51'''<pre>http://www.youtube.com/watch?v=YnJl6qAYdLs</pre><small><small>(if this specific video gets pulled, search for any of the cartoon titles identifying the gag in [http://tvtropes.org/pmwiki/pmwiki.php/Main/XylophoneGag this] article)</small></small>
-The goal was to apply machine learning techniques to digital audio to identify a player's mistakes playing a known melody.  This is a course project[http://www.music.informatics.indiana.edu/courses/I547/] in only about 500 lines of code, so there are severe restrictions on its general utility.  It works on monophonic, single instrument melodies, and works better if the melody is short and played on a wind instrument.  It does not identify timing errors and wasn't tested on skipped notes, only wrong notes.
+The goal was to apply machine learning techniques to digital audio to identify a player's mistakes playing a known melody.  This is a course project[http://www.music.informatics.indiana.edu/courses/I547/] in only about 500 lines of code, so there are severe restrictions on its general utility.  It works on monophonic, single instrument melodies, and works better if the melody is short and played on a wind instrument.  It does not identify timing errors and wasn't tested on skipped notes, only wrong notes.  To keep close roots to the gag, I only benchmarked the code on the classic tune for the Irish poem, "[http://en.wikipedia.org/wiki/Believe_Me,_if_All_Those_Endearing_Young_Charms Believe me, if all those endearing young charms]."[http://www.contemplator.com/ireland/believe.html]
-To keep close roots to the gag, I only benchmarked the code on the classic tune for the Irish poem, "[http://en.wikipedia.org/wiki/Believe_Me,_if_All_Those_Endearing_Young_Charms Believe me, if all those endearing young charms]."[http://www.contemplator.com/ireland/believe.html] However, I do demonstrate the robustness of the code to identify mistakes across the melody, as played in different keys and across a few instruments.  For my simple test examples, and given the limits of the HMM that I built the code on, this system avoids both false positives and negatives (thinking there is a mistake when there isn't and thinking there is not a mistake when there is).
+However, I do demonstrate the robustness of the code to identify mistakes across the melody, as played in different keys, at different tempos, and across a few instruments.  For my simple test examples, and given the limits of the HMM that I built the code on, this system avoids both false positives and negatives (thinking there is a mistake when there isn't and thinking there is not a mistake when there is).
-I modified a simple HMM implementation[http://www.music.informatics.indiana.edu/courses/I547/hw6_sln.r][http://www.music.informatics.indiana.edu/courses/I547/] written in [http://r-project.org R] to have rudimentary score following.  I then introduced an obnoxious screech into the audio to indicate identification of each mistake.
+I modified a simple HMM implementation[http://www.music.informatics.indiana.edu/courses/I547/hw6_sln.r][http://www.music.informatics.indiana.edu/courses/I547/] written in [http://www.r-project.org R] to have rudimentary score following.  I then introduced an obnoxious screech into the audio to indicate identification of each mistake.
 ==the tune==
 [[Image:charms_sheet.jpg|right|400 px|thumb|first page of Endearing Charms score[http://www.musicnotes.com/sheetmusic/mtdVPE.asp?ppn=MN0062193].  This project uses first five measures.]]
-[[Image:midi_notes_key.GIF|right|200 px|thumb| midi cheat sheet[http://www.phys.unsw.edu.au/jw/notes.htm]]]
+[[Image:midi_notes_key.GIF|right|200 px|thumb| midi cheat sheet[http://www.phys.unsw.edu.au/jw/notes.html]]]
 ----
 ==code==
 ===the code===
 [[xylophone gag project code]]
-[http://enfascination.com/forever/xylophonegag/endearing_charms_code006.r]
+[http://enfascination.com/forever/xylophonegag/endearing_charms_code_006.r]
 ===modifications===
 I added a few things to the HMM implementation here [http://www.music.informatics.indiana.edu/courses/I547/hw6_sln.r]
@@ Line 102: / Line 107: @@
 #[http://enfascination.com/forever/xylophonegag/FinalYosemiteOriginal.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalYosemiteConverted.wav Recognized representation]
-#[http://enfascination.com/forever/xylophonegag/FinalYosemiteMarkedUp.wav Error-marked representation](notice that there are no identified errors)
+#[http://enfascination.com/forever/xylophonegag/FinalYosemiteMarkedUp.wav Error-marked representation](notice little errors on piano attack)
+|
-[[Image:FinalYosemiteConverted.jpg|right|400px|thumb|notice much lower key]]
+[[Image:FinalYosemiteConverted.jpg|right|400px|thumb|notice much lower key and faster play]]
 |}

@@ Line 9: / Line 9: @@
 ==the tune==
-[[Image:charms_sheet.jpg|right|400 px]][http://www.musicnotes.com/sheetmusic/mtdVPE.asp?ppn=MN0062193]
+[[Image:charms_sheet.jpg|right|400 px|thumb|first page of Endearing Charms score[http://www.musicnotes.com/sheetmusic/mtdVPE.asp?ppn=MN0062193].  This project uses first five measures.]]
-−
 [[Image:midi_notes_key.GIF|right|200 px|thumb| midi cheat sheet[http://www.phys.unsw.edu.au/jw/notes.htm]]]
 ----
@@ Line 50: / Line 49: @@
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos41112MarkedUp.wav Error-marked representation]
+|
-[[Image:FinalErrorPos41112Converted.jpg|right|400 px]]
+[[Image:FinalErrorPos41112Converted.jpg|right|400 px|thumb|note detection error at time 200. Representation is flat.]]
 |}
 ====other mistake (on recorder)====
@@ Line 56: / Line 55: @@
 |- border="0"
+|
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos2356Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos2356Converted.wav Recognized representation]
@@ Line 83: / Line 83: @@
 [[Image:FinalNoErrors02Converted.jpg|right|400 px]]
 |}
 ----
 {| cellpadding="2" style="border: 0px;"
@@ Line 93: / Line 94: @@
 [[Image:FinalNoErrors03Converted.jpg|right|400px]]
 |}
 ----
 {| cellpadding="2" style="border: 0px;"
@@ Line 114: / Line 116: @@
 #[http://enfascination.com/forever/xylophonegag/FinalBugsBunny02MarkedUp.wav Error-marked representation]
+|
-[[Image:FinalBugsBunny02Converted.jpg|right|400px]]
+[[Image:FinalBugsBunny02Converted.jpg|right|400px|thumb|notice all the detected silence, and sparsity of detected notes]]
 |}
 ----

@@ Line 9: / Line 9: @@
 ==the tune==
-===sheet music===
+[[Image:charms_sheet.jpg|right|600 px]][http://www.musicnotes.com/sheetmusic/mtdVPE.asp?ppn=MN0062193]
 [[Image:charms_sheet.jpg|right]][http://www.musicnotes.com/sheetmusic/mtdVPE.asp?ppn=MN0062193]
-===midi representation===
+[[Image:midi_notes_key.GIF|right|200 px| [http://www.phys.unsw.edu.au/jw/notes.html midi cheat sheet]]]
-[[Image:midi_notes_key.GIF|right]][http://www.phys.unsw.edu.au/jw/notes.html]
+----
 ==code==
 ===the code===
@@ Line 30: / Line 30: @@
 ==examples==
 ===caught mistakes===
 ====canonical mistake (on penny whistle)====
-[[Image:FinalErrorPos1112Converted.jpg|right]]
+[[Image:FinalErrorPos1112Converted.jpg|right|400px]]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos1112Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos1112Converted.wav Recognized representation]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos1112MarkedUp.wav Error-marked representation]
 ====canonical mistake with detection error====
-[[Image:FinalErrorPos41112Converted.jpg|right]]
+[[Image:FinalErrorPos41112Converted.jpg|right|400 px]]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos41112Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos41112Converted.wav Recognized representation]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos41112MarkedUp.wav Error-marked representation]
 ====other mistake (on recorder)====
-[[Image:FinalErrorPos2356Converted.jpg|right]]
+[[Image:FinalErrorPos2356Converted.jpg|right|400 px]]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos2356Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalErrorPos2356Converted.wav Recognized representation]
@@ Line 48: / Line 49: @@
 ===good versions (no mistakes played and no mistakes found)===
-[[Image:FinalNoErrors01Converted.jpg|right]]
+[[Image:FinalNoErrors01Converted.jpg|right|400 px]]
 #[http://enfascination.com/forever/xylophonegag/FinalNoErrors01Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalNoErrors01Converted.wav Recognized representation]
 #[http://enfascination.com/forever/xylophonegag/FinalNoErrors01MarkedUp.wav Error-marked representation](notice that there are no identifications)
 ----
-[[Image:FinalNoErrors02Converted.jpg|right]]
+[[Image:FinalNoErrors02Converted.jpg|right|400 px]]
 #[http://enfascination.com/forever/xylophonegag/FinalNoErrors02Original.wav Original recording]
 #[http://enfascination.com/forever/xylophonegag/FinalNoErrors02Converted.wav Recognized representation]