SEAL: Self-supervised Embodied Active Learning using Exploration and 3D
  Consistency
  In this paper, we explore how we can build upon the data and models of
Internet images and use them to adapt to robot vision without requiring any
extra labels. We present a framework called Self-supervised Embodied Active
Learning (SEAL). It utilizes perception models trained on internet images to
learn an active exploration policy. The observations gathered by this
exploration policy are labelled using 3D consistency and used to improve the
perception model. We build and utilize 3D semantic maps to learn both action
and perception in a completely self-supervised manner. The semantic map is used
to compute an intrinsic motivation reward for training the exploration policy
and for labelling the agent observations using spatio-temporal 3D consistency
and label propagation. We demonstrate that the SEAL framework can be used to
close the action-perception loop: it improves object detection and instance
segmentation performance of a pretrained perception model by just moving around
in training environments and the improved perception model can be used to
improve Object Goal Navigation.
