A 3D-Deep-Learning-based Augmented Reality Calibration Method for
  Robotic Environments using Depth Sensor Data
  Augmented Reality and mobile robots are gaining much attention within
industries due to the high potential to make processes cost and time efficient.
To facilitate augmented reality, a calibration between the Augmented Reality
device and the environment is necessary. This is a challenge when dealing with
mobile robots due to the mobility of all entities making the environment
dynamic. On this account, we propose a novel approach to calibrate the
Augmented Reality device using 3D depth sensor data. We use the depth camera of
a cutting edge Augmented Reality Device - the Microsoft Hololens for deep
learning based calibration. Therefore, we modified a neural network based on
the recently published VoteNet architecture which works directly on the point
cloud input observed by the Hololens. We achieve satisfying results and
eliminate external tools like markers, thus enabling a more intuitive and
flexible work flow for Augmented Reality integration. The results are adaptable
to work with all depth cameras and are promising for further research.
Furthermore, we introduce an open source 3D point cloud labeling tool, which is
to our knowledge the first open source tool for labeling raw point cloud data.
