PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
  3D neural networks are widely used in real-world applications (e.g., AR/VR
headsets, self-driving cars). They are required to be fast and accurate;
however, limited hardware resources on edge devices make these requirements
rather challenging. Previous work processes 3D data using either voxel-based or
point-based neural networks, but both types of 3D models are not
hardware-efficient due to the large memory footprint and random memory access.
In this paper, we study 3D deep learning from the efficiency perspective. We
first systematically analyze the bottlenecks of previous 3D methods. We then
combine the best from point-based and voxel-based models together and propose a
novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We
further enhance this primitive with the sparse convolution to make it more
effective in processing large (outdoor) scenes. Based on our designed 3D
primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the
best 3D network architecture given a resource constraint. We evaluate our
proposed method on six representative benchmark datasets, achieving
state-of-the-art performance with 1.8-23.7x measured speedup. Furthermore, our
method has been deployed to the autonomous racing vehicle of MIT Driverless,
achieving larger detection range, higher accuracy and lower latency.
