SE(3)-Equivariant Attention Networks for Shape Reconstruction in
  Function Space
  We propose the first SE(3)-equivariant coordinate-based network for learning
occupancy fields from point clouds. In contrast to previous shape
reconstruction methods that align the input to a regular grid, we operate
directly on the irregular, unoriented point cloud. We leverage attention
mechanisms in order to preserve the set structure (permutation equivariance and
variable length) of the input. At the same time, attention layers enable local
shape modelling, a crucial property for scalability to large scenes. In
contrast to architectures that create a global signature for the shape, we
operate on local tokens. Given an unoriented, sparse, noisy point cloud as
input, we produce equivariant features for each point. These serve as keys and
values for the subsequent equivariant cross-attention blocks that parametrize
the occupancy field. By querying an arbitrary point in space, we predict its
occupancy score. We show that our method outperforms previous SO(3)-equivariant
methods, as well as non-equivariant methods trained on SO(3)-augmented
datasets. More importantly, local modelling together with SE(3)-equivariance
create an ideal setting for SE(3) scene reconstruction. We show that by
training only on single objects and without any pre-segmentation, we can
reconstruct a novel scene with single-object performance.
