Monika Wernerus
Jonathan Diehl
The Visual Sequencer allows the user to lay out music patterns as if they consistet of tokens of a board game. By adding a token the corresponding note or drum is played at the associated time. By laying out music visually even the inexperienced user can quickly come up with something that actually sounds like music. Also the user learns what combinations of notes sound harmonic by moving around tokens.
The visual sequencer is implemented in Max/MSP by cycling '74. At first we had to get the video camera to report information about the placement of tokes on the board. Then we had to design an iterator to walk through the rows of the grid and report the notes. These notes must then be translated into midi tones.
The only real challenge was to get the visual recognition of the tokens working. First we needed a tool to calibrate the camera. Then the resolution had to be reduced according to our grid size. Then the color information needed to be extracted to make recognition reliable. Jitter provided several patches which helped with these tasks, like the RGB splitter. We also tryed to use cyclops but gave up since we would have had to define 256 zones which was really inefficient and annoying to handle.
Additional features we thought about include adding visual controls for instrument settings like velocity and duration of notes, offering major and minor scales, adding the possibility to play drums and notes and of course improved vision handling like ignoring the image of the hand.
data-controller Patch for visual processing
iSight Patch for iSight calibration
midi-patterns Patch for iterator and midi playback
drums Table for for midi drums
piano Table for midi notes