Introduction
Usually creating music involves using your mouth, arms or digits. Wouldn't it be an interesting experience to create music with just your feet for a change? By using a dance mat, you can select the rhythms of your choice and can accompany the background beats with your own musical composition. To make compositing your own music even more rewarding, our input device reacts to certain input sequences of yours (also known as "Combos" in professional circles), and adds its own little tunes. Sounds interesting? Keep on reading for a more detailed overview...
Motivation
Usually, while listening to music playing, people react with
feedbacks, either consciously or subconsciously..
This feedback includes stepping the feet on the floor or
moving the, the foot up and down. Another more complex feedback would
be moving the feet harmonically with the same speed as the music --
basically dancing at its simplest form.
By considering this natural reaction of human beings we thought of
getting use of the Dance Mat, which fits this natural
concept and also includes having fun and excitement while jumping up
and
down and hearing the produced music from it.
The device
The Hop-A-Tone musical instrument uses a standard issue dance
mat - available on E-Bay for only a couple of bucks:
The model we use has 10 buttons: The SELECT and START buttons, and the
8 action buttons that form a circle around the center.
Usage
We have decided to go with a 2-modal input. Another possible input method is also described, but was discarded. A very early implementation exists, though.
2-Modal Mode
In this input method the dance mat works in two modes, which
you can toggle by using the SELECT button:
When in rhythm mode, you can switch between different background
rhythms simply by stepping on one of the 8 action buttons. The rhythm
will keep playing until you select the currently playing rhythm again.
When in compose mode, you can play the different keys C,D,E,F,G,A,H,C
by stepping on one of the 8 action buttons. If you want to play a
chord, just push two buttons belonging to one C-Major chord at the same
time. The device will then add the third note by itself.
Mixed Mode (implemented, but not used)
Step in the middle of the mat. Now, step on a button
of your choice with both feet. This selects the background rhythm you
will be using. Each of the 8 buttons plays a different beat. Should you
wish to change the background, simply put your feet on another button.
Should you ever wish to stop any rhythm, place both your feet in the
center of the mat.
Should you wish to change the rhythm on the fly, just step with one
foot to the button triggering the new rhythm, and then lift the other
foot from the previous button. The rhythm will then change to the your
newly selected one.
While keeping one foot on the button which selects your background
rhythm, use the other one to play notes. The note which gets played is
always relative to your own position: From your rhythm-button
clock-wise, the buttons play the notes C, D, E, F, G, A, H.
Don't worry about keeping the rhythm running: As long as you keep one
foot on the button, it will keep playing.
While this method of input sounds cool in theory, it's quite confusing
in practice and unnecessary since you don't really change the
background that often during composition.
Completing combos
To make the dancing experience a little bit more rewarding and to convey a feeling of accomplishment, it is possible to program combos into our instrument. This means that once you've completed one of a few specific sequences of notes within a certain time period, the instrument will react by "answering" to your input. Why don't you try it out yourself?
Combo | Reaction |
---|---|
E, G, C2 | Ha Do Ken! |
A, G, D, D | Speed of Sound |
Challenges
A rough outline of our milestones:
- brainstorming sessions, designing the first ideas
- Find functional USB driver for the dance mat
- Java & Max/MSP understanding
- Implementing simple prototypes in JAVA & Max/MSP Patches
- Usertests (in general with ourselves, but when possible with persons who are not involved (Thank you Eugen!)
- Refinement of prototype (in special: mappings, delays, instruments, chords & scale)
- Usertests
- Latest prototype version
In our first brainstorming session, we came up with fantastic ideas
like
if you hop a combo, the computer finishes the music
sequence you started, or the background rhythm is adapted to the speed
of your
played music. But soon we realized these thoughts were
very "mission impossible". Therefore we concentrated on our
main goal: producing music and not producing wonderful sounding music
;-)
From a technical point of view, we needed a special USB driver
to get the mat running, since the Macs did not identify our device
correctly. We looked for a third-party alternative and stumbled
over "USB Overdrive". This driver maps the
mat-buttons to keys from the keyboard.
In class we got a short introduction to Max/MSP and all of us had
programming experiences with Java. But we had to learn how to connect
external java classes to Max/MSP. Our approach was to do the main logic
part in Java and the MIDI signals in Max/MSP.
Our first protoypes arised from playing around with Java and Max/MSP.
With the "trial&error" method we went along and implemented our
first prototypes.
In the following usertests we became aware of our mistakes. We had to
adjust the reaction time and delays according to human capabilities.
The usability of a dance mat is completely different from typing on a
keyboard. And this difference doesn't consist only of some ten
miliseconds... So during our user-self-tests we did incremental changes
to our prototype.
For further refinement, we put more thought into the
actual mappings. On
the one hand we had the very limited user input of the mat (8+2
digital buttons), and on the other hand the high variety of an
instrument. The speed of the music should of course be proportional to
the
speed of pressing the buttons on the mat.
One of our biggest
problems was the arrangement of the notes: what arrangement
feels the most
natural? What is the starting position of a user -- in the middle of
the
mat or right in front of the mat? We implemented two solutions and
came to the conclusion that the tones should be arranged from left to
right,
buttom to top according to their pitch. The button in the upper-right
corner of the mat has the highest pitch.
To make our instrument more versatile, yet still simple to
use, we added chords as well. Since it's very
impractical to
the hands and humans have no third
leg (except for a lucky few), we decided to add chords of the major
scale in the way, that if
two notes are pressed, the third chord tone is added by the computer.
In the upcoming usertest, when we were actuallytrying to play a song on
our instrument, we had to adjust the tolerance of our device to detect
a chord according to human reaction times
Our latest prototype now maps to each button a full tone from C to C
and
gives the user the possibility to play some chords of the major scale
if two buttons are pressed together.
Future work
To improve and upgrade Hop-A-Tone we thought about a group
experience. Creating music is much more fun in a group, therefore
connecting several dance mats to a computer to get a band
of hopping artists sounds like a blast! Each musician
could play his
favourite instrument
and all together compose a common score.
Another approach could be to add another class of sensors. For
example, an
observing camera could produce some rhythms or tones if the actor does
a special movement with his arms or if he loses ground contact (=
jumps).
A better way to integrate the background rhythm
would certainly go a long way to improve the experience.
Some kind of visualization could be implemented as well. The produced
sound could be displayed in curves like an oscillograph does.
One could also think about a dance mat with more buttons to
address a bigger
scale of tones and/or to mix several instruments.
References
Perhaps you wonder why anybody would have such a mat at
home??! Here is a possible answer...
http://en.wikipedia.org/wiki/Dance_Dance_Revolution
First of all, we needed a driver for the dance mat to map the buttons
to keys from the keyboard
http://www.usboverdrive.com/
Since we are all unexperienced in doing music, we needed a short
introduction of scales and chords
http://www.looknohands.com/chordhouse/piano/
For our second prototype we chose a different instrument: the guitar.
But we recognized piano chords sound terrible on a guitar and therefore
we looked for guitarchords
http://www.guitaretab.com/
To implement the Max/MSP Patch, we needed the corresponding MIDI
settings to the pitch of the used notes.
http://www.dancetech.com/aa_dt_new/tables/pitch-freq-note.cfm
Downloads
Here you can test our prototype on your own.
Max/MSP Patch
Java Stuff
Installation & Running Instruction
You have to copy the Java classes into your MAX/MSP
classfile-directory (on Windows usually c:\Program Files\Common
Files\Cycling '74\java\classes). You can install the patch itself in
any directory.
When running, remember to enable the toggle button setting the
groove-object to loop. An empty Java window will popup, which you have
to give focus to process the input. Please note that while keyboard
input is possible using the keypad and the /-button for SELECT, there
are problems with press/release events due to the driver hacking
with USB Overdrive.
Should you want to use the Event Heap or Debug Version, edit the file
'dance_main' to use the output you want and start it as a standalone
Java application. Please note that the Event Heap version is not well tested.
If you have some improvements, please let us know.
( Remove the NOSPAM in the adresses )
Mail
to Mahsa, Christian and David
Implementation Overview
Java part:
Combo.java | Implements object which contains state for one combo |
ComboManager.java | Handles Combo-recognition |
ComboStorage.java | Static class, combo data is loaded from here |
dance_main.java | Main class for Debug and Event Heap Output |
Dance.java | Handles the actual event input and processing. Generates the output data. |
DanceEHMax.java | Use this class as object in Max/MSP if you receive events by Event Heap |
DanceEHUtils.java | Constants used in Event Heap Output |
DanceOutput.java | Interface which specifies device-output |
DanceOutputDebug.java | Implements DanceOutput; Only console information |
DanceOutputEH.java | Implements DanceOutput; Sends events through Event Heap |
DanceOutputMax.java | Implements DanceOuput; Use this class as object in Max/MSP to receive the device data |