The Sound of Juggling

We would like to propose the creation of a computational instrument that is not only able to act as a frontend to any possible instrument, but also provides the observer with a pleasing visual presentation.

Sound Creation

The users will have coloured objects in their hands, whereas each colour is mapped to a particular instrument. Moving those objects around, the user can create a ``Music Context" which can be compared to the graphics context that OpenGL and similar systems use. The absolute positions of the objects are captured by a camera. In order to trigger output, the users have to step. Sensors on their shoes recognise the movement and create sound with the parameters of the Music Context. The sound will be created as MIDI output.

The mapping of beat to steps is very natural, as it is used for dancing. While dancing, people move their feet over a repetitive pattern which is constant apart from slight variations.

Mappings
There are three dimensions that have to be taken care of. First of all, the choice of the instrument needs to be specified. In our system, we will assign each instrument a fixed colour. In order to switch to another instrument, the user will have to take another object. This is an usual action for musicians and thus a natural mapping.

The second point concerns manipulation the pitch. Sheets have notes with a higher pitch located above those that are lower. We will map the notes the same way. Holding the object higher will increase the pitch.

Last but not least, we have to do some manipulation of the volume. The volume can be mapped to the horizontal axis, choosing a centre point at the user's centre. We will assume that the camera points to the middle of the user's body.

Applications

There are several ways of using our system. The first might yield the answer to a question that is at least as important as the one of life, the universe, and everything. ;-)
How does juggling sound?
This question which, after "How is the sound of one Hand clapping?", must clearly be the most asked question in existence, can finally be solved. Each of the balls will be mapped to a certain instrument and, depending on the required effect, the performer can either lock the trigger or activate the triggers at different moments providing the audience with an unforgettable acoustic experience.
Drawing music
A performer can, using objects in each hand, draw the flow of the music into the air while triggering the instrument with his feet. The audience will not only be able to hear, but to see the movements of the melody.
Dancing
Dancing is one of most natural things of following the rhythm. In our system, we turn this relationship around, letting the dancer create the rhythm. The instruments can be defined by coloured spots on the back of the dancers. Turning around increases the volume until it suddenly disappears, and bending your knees leads to the typical beginning of a tango.
Collaborative music-making
If the distance between the camera and the performing stage is great enough, multiple people can work together in the creation of the music circumventing the natural limitations of simultaneous instrument per person. The collaboration effect is enhanced due to the mapping of the volume to the x-axis which forces the performers to switch places or instrument to change the relative volume of their instruments.

Extensions and Variations