Face of Anna Koster

Anna Koster

Student Assistant

This email address is being protected from spambots. You need JavaScript enabled to view it.

Project Involvement

Diploma Thesis with advisor Thorsten Karrer: "WiiCon: Acceleration Based Real-Time Conducting Gesture Recognition for Personal Orchestra"

Personal Orchestra is a research project which allows laymen to conduct an audio
and video recording of an orchestra. The orchestra adapts to the tempo of the conductor
and plays in the given speed. Up to now, a robust gesture recognition of
authentic conducting gestures for professional conductors was missing.
In this thesis, a new real-time gesture recognition plug-in for Personal Orchestra
is developed, which is able to process three-dimensional acceleration data of the
NintendoWii remote.

Due to the time-based character of conducting gestures, the hidden Markov model
is chosen to solve the task of automatic learning and recognition of 1-beat to 4-beat
To achieve good recognition results, a large amount of training data is needed. For
this reason, two user studies for data acquirement were organized. Over 16000
conducting bars were recorded from 70 differing participants of the studies.
The hidden Markov model was trained with these conducting gestures for each
type of bar separately with the help of dynamic programming. After some
iterations of time alignment, the attributes in the training phase were optimized
for the recognition phase.

The recognition phase of conducting gestures works in real-time with unknown
conductors. Some adjustments were made in the recognition algorithm, so that the
recognition works while the conducting is executed. It is possible to recognize at
which position the conductor is in the bar at each time-point.
The hidden Markov model was trained with half of the conductors, the recognition
phase was evaluates with the other half, respectively. The correct position of beats
in a given type of bar is recognized correctly up to a rate of 92%.
Additionally a second task was solved: Where to recognize the beats by the system?
We identified the beats at the maximum peaks of the acceleration data, especially
in the down and forward acceleration of theWii remote.



  • Anna Koster. WiiCon:Acceleration Based Real-Time Conducting Gesture Recognition for Personal Orchestra. Diploma Thesis, RWTH Aachen University, Aachen, December 2008.
    PDF DocumentBibTeX Entry
  • Elaine M. Huang, Anna Koster and Jan Borchers. Overcoming Assumptions and Uncovering Practices: When Does the Public Really Look at Public Displays?.  In Pervasive 2008, Lecture Notes in Computer Science, pages 228-243, Springer Link, Sydney, Australia, May 2008.
    HomepagePDF DocumentBibTeX Entry

We use cookies on our website. Some of them are essential for the operation of the site, while others help us to improve this site and the user experience (tracking cookies). You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.