Project AURA
AURA: Ad-Hoc Interaction With Non-Planar Objects
This project explores how to make everyday objects and surfaces, including non-planar ones, in our environments into interactive controllers. The project is funded by a 300k€ grant from the German National Science Foundation (DFG), and runs from June 2012 to May 2015.
People
Christian Corsten, M.Sc. (contact)
Dipl.-Inform. Dipl.-Wirt.-Inform. Max Möllers (alumnus)
Prof. Dr. Jan Borchers
Projects
Co-Optjects: Instant User Interfaces Through Everyday Objects
Master's Thesis by Christian Corsten, M.Sc. (download thesis)
Abstract Inspired by the process of appropriation of everyday objects (e.g., a knife) as workaround for replacing specially designed tools (e.g., a letter opener), we transferred this paradigm to the field of digital interfaces. We introduce a new interaction metaphor, called Instant User Interfaces (IUIs), which enables people to spontaneously use everyday artifacts as temporary replacements in case the dedicated input device is missing.
As initial approach, we conducted a diary photo study for determining which objects are perceived “everyday” and therefore come into consideration for spontaneous appropriation. Next, in the scope of a Wizard of Oz study, we tested hypothetic appropriation of these objects for various digital interfaces encountered daily. For each scenario, we identified object-gesture-action mappings as devised by the testers. We found out that concerning discrete input, users tended to imitate a dedicated controller as well as possible with everyday objects. Concerning continuous input, however, people created a physical instance of the target itself, denoted by Physical Instantiation (PI). In a final study, we tested everyday object interaction with an interactive prototype technically based on a combination of depth camera and fiducial tracking for turning passive objects into physical controllers. Besides testing predefined mappings based on the results of the previous study, we examined the interplay of normal object usage with appropriation.
The working prototype showed that the idea of IUIs is promising as appropriation of everyday objects helps managing simple tasks without being dependent on
dedicated controllers. Despite interaction flaws that still exist due to technical limitations, we see the foundation laid for this new interaction metaphor. Motivated by these results, we present a list of guidelines based on the experience collected in the user studies that serves as a primer to future IUI research.
A framework for inexpensive and unobtrusive tracking of everyday objects and single-touch detection
Master's Thesis by Ignacio Avellino, M.Sc. (download thesis)
Abstract Technology development has surrounded people’s daily life with devices, which, for people’s convenience, include most of the time a dedicated controller that allows operating them remotely. A problem arises when such controllers are temporarily unavailable or out of a person’s reach. Instant User Interfaces (IUIs) allow people to casually reappropriate an everyday object at their reach and use it as an input device. In order for a system to interpret such interaction, a system capable of detecting object movement in 3D space as well as touches on its surface is needed.
This work aims at the implementation and evaluation of an object tracking and single-touch sensing system, which can operate without intruding into the user’s space. By using a 3D digital model of a real world object (e.g., obtained through an off-the-shelf 3D scanner) it is possible to: (a) automatically and reliably recognise and track objects; (b) identify user actions performed on such objects such as movement and finger touches; and (c) assign semantic value to such actions.
A tracking evaluation performed using data obtained by a state-of-the-art tracking system showed that changes in position were tracked successfully with small errors. Performance on orientation estimation error was found to be higher, although still acceptable. An evaluation on touch accuracy showed how the system performs when the size of touch enabled areas changes in an object.
The presented system enables further research on Instant User Interfaces by allowing investigators to understand interaction with everyday objects without interrupting users.
Publications
- Jan Borchers, Christian Corsten, Max Möllers and Simon Voelker. Von der Fläche zur Kurve: Multi-Touch auf beliebigen Oberflächen und Objekten. In Multi-Touch: Interaktion durch Berührung, Xpert.press, pages 369–392, Springer Verlag, Incollection, February 2014.
- Christian Corsten, Ignacio Avellino, Max Möllers and Jan Borchers. Instant User Interfaces: Repurposing Everyday Objects as Input Devices. In ITS '13: Proceedings of the ACM Symposium on Interactive Tabletops and Surfaces, pages 71–80,October 2013.
- Ignacio Avellino. A Framework for Inexpensive and Unobtrusive Tracking of Everyday objects and single-touch detection. Master's Thesis, RWTH Aachen University, Aachen, April 2013.
- Christian Corsten, Chat Wacharamanotham and Jan Borchers. Fillables: Everyday Vessels as Tangible Controllers with Adjustable Haptics. In CHI '13: Extended Abstracts of the 31st Annual Symposium on Human Factors in Computing Systems, pages 2129–2138,April 2013.
- Christian Corsten. Co-Optjects: Instant User Interfaces Through Everyday Objects. Master's Thesis, RWTH Aachen University, Aachen, January 2012.