Our goal was to create a possibility to visualize MIDI-Events in real-time.
To achieve this, we created a Max/Jitter-Patch that analyzes incoming MIDI data and maps note-on- and note-off-events, pitches, velocities and program changes to different visual effects.
The display window of the patch shows an arrangement of different instruments that can be simulated by a MIDI synthesizer - of course not all of them, only a few selected ones - and highlights the currently selected one.
While a note is played, an animated image is displayed over the corresponding instrument for as long as the note is held down. The colour of the animated “note” is determined by the played pitch, every fullstep (from C to D, D to E etc) there is another colour. Additionally, the octave the pitch belongs is visualized through brightness and saturation of the animation (low brightness and high saturation for low octaves et vice versa).
The velocity influences the size of the animation, of course a bigger size means a higher velocity.
Due to perfomance and other reasons, the visualisation is limited to 3 simultaneously played notes.
With this program, it is possible to e.g. use a MIDI keyboard to play music and get a “live” visualisation of what instrument you are playing and how you are playing it, therefore making it possible for a deaf or hearing-impaired person to “see” the music. Also, this program may be used to teach children how different instruments sound, what they are called etc. in a very playful, hands-on way.