Developing for ARPen

How is the pen tip tracked in the fully-functional ARPen?

In the fully-functional ARPen, one or more markers' position and orientation is used to determine the pen tip. (The pen tip is determined in a more robust manner when multiple markers are identified in the scene.) OpenCV2 is used to detect the arUco markers; see the function findMarker() in OpenCV/OpenCVWrapper.mm for details of marker detection. Note: The markerSize property can be used to adjust the size of the marker (in metre) you want to be detected. The six faces of the ARPen marker box are mapped to an enumeration with names as shown below:

In every frame, based on the detected marker, ARPen calculates the pen tip. When multiple markers are identified, the position of the pen tip is averaged. For the pen tip calculation implementation, see the function calculatePenTip() in MarkerBox.swift.

How is the pen tip calculated?

We use trigonometry to determine the pen tip from the position and orientation of the markers in the marker box. Consider the following image:

To calculate position C from A, we use the following vector:

For markers .Right, .Top, and .Back, the marker is not placed at the center of the face; this is because the handle of the pen is glued to the cube in these markers. Therefore, pen tip calculation for these markers is adjusted to:

How to add your own plugins?

A plugin is a functionality you want to add to the ARPen app. You could e.g., create plugins to build a cube or Please have a look at the guide for Writing Plugins for detailed instructions.

Note that since aruco and opencv2 frameworks do not support x86 architecture, you won't be able to use the Xcode iPhone Simulator for debugging.