TWEND: Twisting and Bending as new Interaction Gesture in Mobile Devices
New: Radio broadcast describing the Twend Demonstrator that was also on board the MS Wissenschaft. This broadcast aired on WDR, RBB.
The UMIC report about the demonstrator onboard the MS Wissenschaft (English).
The press release about the demonstrator (German).
Article (in German) about TWEND on board the MS Wissenschaft.
The Spanish online branch of the "Deutsche Welle" also recently wrote an article about TWEND.
Video of TWEND in use.
What is TWEND?
TWEND is a prototype for direct-manipulation by deformation in mobile devices.Bendable cellphones and PDAs will not be something special in the future. Because of this, the mobile technology excellence cluster UMIC at the RWTH Aachen University developed TWEND as a means of researching the usability of such devices. Flipping pages in the eBook reader of the future, for example, then will work just the same as flipping pages in a traditional book.
The system consists of a hardware protoype that enables users to perform bending gestures and a Mac OS X software framework that maps these gestures to system events. These events can be configured to mimick scrolling, keystrokes or Apple Script events. This enables a quick way to set up experiments to research the value of different bending gestures as input method for interactive tasks involving a computer system.
Since the software framework is extensible, even setups that cannot be done with only the standard keystrokes, scrolling or Apple Script can be established.
The people behind TWEND
TWEND is a research project by Gero Herkenrath, Thorsten Karrer, Jan Borchers, and others at the Media Computing Group. The project is funded through UMIC DFG Excellence Initiative.CHI 2008 Student Research Competiton Runner-Up
This work was sent to the CHI 2008 Student Research Competition and won the second place. The paper is available here.
Motivation
Although the prototype framework still requires a desktop computer or at least a notebook, the goal of the project is to find new interaction methods for mobile devices. Traditional input methods suffer from certain problems in this context. First, there is a real estate problem between space used for input and space used for output. Screen sizes and number of input buttons are limited to keep the device small. Touchscreens counter this, but can lead to occlusion problems. Tilting introduces a new interaction style by facilitating not just the fingers, but sometimes it can be hard to correctly interpret the users intention with this (just try to read something on the iPhone while lying in your bed...).A bendable device also makes use of the users wrists, i.e. just like tilting it makes use of an additional part of the dextrous system. Because the hands are already "in position" holding the device, no further space for input areas is needed.
Building the Prototype
Before the actual TWEND prototype was build, we constructed a smaller one to figure out whether our choice of sensors was feasible for our needs.We used optical bending sensors to measure the amount of bending done to the device. These are embedded in a layer of foam between two pieces of plastic.
The Sensors
The sensors consist of an LED, a photocell and a fibre cable with an abraded section.
If bent, more or less light is lost at the abraded part of the cable which can be measured at the photocell.
The LEDs and the photocell are connected to seperate circuits to prevent cross influence.
More information on the sensors can be found in this paper: http://stacks.iop.org/MST/13/1523 (Note to readers from outside the RWTH network: This links to an online library you may not have full access to)
The Small Test Prototype
This prototype contains only one sensor and can thus only measure bending along one dimension.
Unlike the lower, the upper plastic plate is not glues to the layer of foam so bending and maintenance is easier.
The Final Prototype
These images show the larger, final prototype. It's 15 x 25 cm large and about 1 cm thick.
On the left you can see the insides, i.e. the sensors embedded in the foam and their wiring. There are eight sensors in total.
The right picture shows one of the possible gestures the prototype can recognize. This specific one we call a "wave". Note that it can only be recognized because the horizontal sensors in the prototype are actually devided in pairs of two instead of one long sensors at the upper and lower side of the device.
The Connection & the Software
The sensor measurement is done by one Bluetooth Arduino (see www.arduino.cc) that then streams the eight values of the sensors to a computer that runs the software doing the gesture recognition and system event generation. We use an Apple Macintosh so far, but aside from the parts of the code that generates the actual system events the software should run on any unix machine. However, we did not yet test that.A First Try
As a first experiment to see how our prototype does and what users think of this new interaction technique, we configured it to control parts of the game "World of Warcraft". Participants were asked to navigate a character through the games 3D environment, one time with our prototype and one time with a standard keyboard. As expected they were faster with the keyboard, but the general feedback on the idea to use bending as interaction technique was very positive. Information on how to improve the prototype and its configuration in future experiments was also useful to us.