← Personal Photonics Project
ARPen: In-Situ 3-D Modeling
| We recently released the ARPen iOS app to the App Store. Check it out! |
Ever wanted to 3d print a dock for your phone? Or maybe a coffee cup holder that can be attached next to your car dashboard? If so, you probably need to measure the dimensions of your phone and dashboard, construct 3d models using software, and then convert them to correct format before 3d printing them. Even then, it might happen that your 3d objects do not look as well as you had expected.
In this project, we investigate techniques that allow you to construct 3d models by using a pen to sketch around existing objects. Yes—3d modeling is now as simple as doodling! Your 3d model snaps to the dimensions of existing objects and you can even view your design in-place using a smartphone to make sure you are happy with your model. Then, you can export the model to 3d print it at your local FabLab!
News
- Our study of different menu techniques for the ARPen system has been presented at MobileHCI 2020 (Oct. 9, 2020)
- At CHI 2020, we presented our evaluation of visualization techniques for the position of the ARPen in relation to other objects in the scene (Apr. 25, 2020)
- ARPen will be presented as a full paper at ACM CHI, the premier international conference on Human-Computer Interaction. (Feb 22, 2019)
- You can now track a pen in 3D using just your smartphone! We have built an ARKit-based iOS app that can track a pen. For instructions, head over to the GitHub repo. (Feb. 20, 2018)
Research Papers
Evaluating Menu Techniques for Handheld AR with a Smartphone & Mid-Air Pen.
Paper at MobileHCI 2020
Heatmaps, Shadows, Bubbles, Rays: Comparing Mid-Air Pen Position Visualizations in Handheld AR.
Paper at CHI 2020
Sources
- Feb., 2018: ARKit-based 3D tracking of a custom pen via an iOS app. Source code for the iOS app, and instructions to run the iOS app and build the custom ARPen can be found here: Source code and documentation (GitHub).
- Dec., 2017: Kinect-based 3D tracking of a 3d printed custom pen (or any object). The algorithm is able to track the 3D coordinates and orientation of an object using a single Kinect 2.0: Source code and documentation (GitHub).
Team
Prof. Jan Borchers: Project Manager
Dr. Simon Voelker: Project Manager
Philipp Wacker, M.Sc.: Computer Vision foundations
Krishna Subramanian, M.Sc.: Community Manager
Publications
- Marvin Bruna. ShARePen: Enabling Spectators to Understand Pointing Operations in Co-Located Handheld AR. Master's Thesis, RWTH Aachen University, Aachen, December 2022.
- Philipp Wacker. Interaction Techniques for Mid-Air Pen Input in Handheld Augmented Reality. PhD Thesis, RWTH Aachen University, Aachen, June 2021.
- Andreas Dymek. ARPen "in the Wild" - Expansion and Evaluation of an In-Situ 3D Modeling Application. Bachelor's Thesis, RWTH Aachen University, Aachen, March 2021.
- Philipp Wacker, Oliver Nowak, Simon Voelker and Jan Borchers. Evaluating Menu Techniques for Handheld AR with a Smartphone & Mid-Air Pen. In Proceedings of 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI '20, pages 10, ACM, New York, NY, USA, October 2020.
- Farhadiba Mohammed. Scaling Objects: Implementation and Evaluation of Scaling Techniques for the ARPen System. Bachelor's Thesis, RWTH Aachen University, Aachen, September 2020.
- Martin Huppertz. Evaluating Guided Drawing Techniques in Handheld Augmented Reality. Bachelor's Thesis, RWTH Aachen University, Aachen, August 2020.
- René Schäfer. Redesigning ARPen: Evaluating Different Marker Placements for Mid-Air Pen Interaction. Master's Thesis, RWTH Aachen University, Aachen, June 2020.
- Philipp Wacker, Adrian Wagner, Simon Voelker and Jan Borchers. Heatmaps, Shadows, Bubbles, Rays: Comparing Mid-Air Pen Position Visualizations in Handheld AR. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI '20, pages 719:1–719:11, ACM, New York, NY, USA, April 2020.
- Jan Benscheid. Making Sense of Lines: Interaction Sequences for 3D Modeling with Mid-Air Sketches. Master's Thesis, RWTH Aachen University, Aachen, October 2019.
- Donna Klamma. Rotating Objects: Implementation and Evaluation of Rotation Techniques for the ARPen System. Bachelor's Thesis, RWTH Aachen University, Aachen, August 2019.
- Philipp Wacker, Oliver Nowak, Simon Voelker and Jan Borchers. ARPen: Mid-Air Object Manipulation Techniques for a Bimanual AR System with Pen & Smartphone. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI '19, pages 619:1–619:10, ACM, New York, NY, USA, May 2019.
- Oliver Nowak. ARMenus: An Evaluation of Menu Interfaces in Pen-based AR Applications on Smartphones. Master's Thesis, RWTH Aachen University, Aachen, February 2019.
- Philipp Wacker, Adrian Wagner, Simon Voelker and Jan Borchers. Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality. In Proceedings of the 6th Symposium on Spatial User Interaction, SUI '18, pages 10, ACM, New York, NY, USA, October 2018.
- Felix Wehnert. Pen-based Drawing in Augmented Reality on Mobile Phones. Bachelor's Thesis, RWTH Aachen University, Aachen, April 2018.
- Philipp Wacker, Adrian Wagner, Simon Voelker and Jan Borchers. Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA '18, pages LBW626:1–LBW626:6, ACM, New York, NY, USA, April 2018.
- Adrian Wagner. Analyzing 3D Sketching Performance on Physical Objects in Augmented Reality. Master's Thesis, RWTH Aachen University, Aachen, April 2018.
- Luisa Hoffmann. Eliciting User Gestures for the Interaction with a Pen-Based 3D Modeling Device. Bachelor's Thesis, RWTH Aachen University, Aachen, March 2017.
2022
2021
2020
2019
2018
2017
Funding
ARPen is part of the Personal Photonics project, which is funded by the German Federal Ministry of Education and Research (BMBF) as part of their Open Photonics call (project ID 13N14065).
