3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep
  Learning
  mmWave radar has been shown as an effective sensing technique in low
visibility, smoke, dusty, and dense fog environment. However tapping the
potential of radar sensing to reconstruct 3D object shapes remains a great
challenge, due to the characteristics of radar data such as sparsity, low
resolution, specularity, high noise, and multi-path induced shadow reflections
and artifacts. In this paper we propose 3D Reconstruction and Imaging via
mmWave Radar (3DRIMR), a deep learning based architecture that reconstructs 3D
shape of an object in dense detailed point cloud format, based on sparse raw
mmWave radar intensity data. The architecture consists of two back-to-back
conditional GAN deep neural networks: the first generator network generates 2D
depth images based on raw radar intensity data, and the second generator
network outputs 3D point clouds based on the results of the first generator.
The architecture exploits both convolutional neural network's convolutional
operation (that extracts local structure neighborhood information) and the
efficiency and detailed geometry capture capability of point clouds (other than
costly voxelization of 3D space or distance fields). Our experiments have
demonstrated 3DRIMR's effectiveness in reconstructing 3D objects, and its
performance improvement over standard techniques.
