Evaluating Menu Techniques for Handheld AR with a Smartphone & Mid-Air Pen
Paper at ACM MobileHCI '20
by Philipp Wacker, Oliver Nowak, Simon Voelker, and Jan Borchers
Abstract
Adding a mid-air pen to Handheld Augmented Reality creates a new kind of bimanual interaction for which many fundamental interaction design questions have not been answered yet. In particular, menus are an essential component in most visual interfaces, but it is unclear how to best interact with them in this setting: using the pen in mid-air or on a surface, using the touchscreen, or by moving the smartphone itself. We compared basic menus for these methods by analyzing success rates, selection times, device movement, and subjective ratings. Our results indicate that interacting with a mid-air menu using the pen, and operating a menu with the hand holding the smartphone, are sufficiently competitive to the current standard of two-handed touchscreen interaction, so that interaction designers can freely choose among them based on the interaction context of their application.
Video
Software and Data
These are explanations for the supplementary materials provided of the MobileHCI paper: "Evaluating Menu Techniques for Handheld AR with a Smartphone & Mid-Air Pen".
There are three sets of materials provided.
01 Software: You can build the application for iOS using Xcode. Print the pdf "aruco-marker" to use as a simple ARPen for testing the menu techniques. To use the techniques, select a menu technique from the menu on the left. The scene will display a number of cubes and a target emoji in the top right corner of the screen. Buttons on the top left allow you to perform a button press. Note that this button overlaps with the one-handed menu but it would not exist if a full ARPen is connected. You can toggle the menu on the left by performing a three-finger swipe down gesture. |
01-Software.zip |
02 Evaluation:
To perform the evaluation, run the python scripts "createDataTablesAndInitialPlots.py" and "createDataTablesAndInitialPlotsQual.py". These will prepare the data tables and perform bootstrapping to calculate the confidence intervals. The results are stored in a new folder "/DataTables/" with either a "Data" or "CI" prefix to the filename. Based on these data tables, the scripts also prepare initial plots for the results. These are stored in the folder "/Figures/". Note: the scripts require Python3, numpy, pandas, and the ARCH library for bootstrapping (https://pypi.org/project/arch/). |
02-Evaluation.zip |
03 Alternative Analysis: |
03-Alternative-Analysis.pdf |
Publications
- Philipp Wacker, Oliver Nowak, Simon Voelker and Jan Borchers. Evaluating Menu Techniques for Handheld AR with a Smartphone & Mid-Air Pen. In Proceedings of 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI '20, pages 10, ACM, New York, NY, USA, October 2020.