Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
  3D object representation learning is a fundamental challenge in computer
vision to infer about the 3D world. Recent advances in deep learning have shown
their efficiency in 3D object recognition, among which view-based methods have
performed best so far. However, feature learning of multiple views in existing
methods is mostly performed in a supervised fashion, which often requires a
large amount of data labels with high costs. In contrast, self-supervised
learning aims to learn multi-view feature representations without involving
labeled data. To this end, we propose a novel self-supervised paradigm to learn
Multi-View Transformation Equivariant Representations (MV-TER), exploring the
equivariant transformations of a 3D object and its projected multiple views.
Specifically, we perform a 3D transformation on a 3D object, and obtain
multiple views before and after the transformation via projection. Then, we
self-train a representation to capture the intrinsic 3D object representation
by decoding 3D transformation parameters from the fused feature representations
of multiple views before and after the transformation. Experimental results
demonstrate that the proposed MV-TER significantly outperforms the
state-of-the-art view-based approaches in 3D object classification and
retrieval tasks, and show the generalization to real-world datasets.
