ARPen: In-Situ 3-D Modeling

[Main Project Page]

Ever wanted to 3d print a dock for your phone? Or maybe a coffee cup holder that can be attached next to your car dashboard? If so, you probably need to measure the dimensions of your phone and dashboard, construct 3d models using software, and then convert them to correct format before 3d printing them. Even then, it might happen that your 3d objects do not look as well as you had expected!

In this project, we investigate techniques that allow you to construct 3d models by using a pen to sketch around existing objects. Yes—3d modeling is now as simple as doodling! Your 3d model snaps to the dimensions of existing objects and you can even view your design in-place using a smartphone to make sure you are happy with your model. Then, you can export the model to 3d print it at your local FabLab!


  • Feb. 20, 2018: You can now track a pen in 3D using just your smartphone! We have built an ARKit-based iOS app that can track a pen. For instructions, head over to the GitHub repo. The ARPen app will be available on the iOS App Store soon.


  • Feb., 2018: ARKit-based 3D tracking of a custom pen via an iOS app. Source code for the iOS app, and instructions to run the iOS app and build the custom ARPen can be found here: Source code and documentation (GitHub).
  • Dec., 2017: Kinect-based 3D tracking of a custom object. The algorithm is able to track the 3D coordinates and orientation of an object using a single Kinect 2.0: Source code and documentation (GitLab).


Prof. Jan Borchers: Project Manager
Krishna Subramanian, M.Sc.: Community Manager
Dr. Simon Voelker: Project Manager
Philipp Wacker, M.Sc.: Computer Vision foundations



Philipp Wacker, Adrian Wagner, Simon Voelker, and Jan Borchers. Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI EA '18, pages LBW626:1–LBW626:6, New York, NY, USA, April 2018. ACM. Homepage PDF Document BibTeX Entry.
Felix Wehnert. Pen-based Drawing in Augmented Reality on Mobile Phones. Bachelor's thesis, RWTH Aachen University, Aachen, April 2018. PDF Document BibTeX Entry.


Luisa Hoffmann. Eliciting User Gestures for the Interaction with a Pen-Based 3D Modeling Device. Bachelor's thesis, RWTH Aachen University, Aachen, March 2017. PDF Document BibTeX Entry.

Internal project pages (registered users only)


This project is sponsored by the German Federal Ministry of Education and Research (BMBF) as part of their Open Photonics call (Personal Photonics, 13N14065).

Created by subramanian. Last Modification: Wednesday 21 of March, 2018 11:03:25 by subramanian.

Media Computing Group at RWTH Aachen