ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
  Objects play a crucial role in our everyday activities. Though multisensory
object-centric learning has shown great potential lately, the modeling of
objects in prior work is rather unrealistic. ObjectFolder 1.0 is a recent
dataset that introduces 100 virtualized objects with visual, acoustic, and
tactile sensory data. However, the dataset is small in scale and the
multisensory data is of limited quality, hampering generalization to real-world
scenarios. We present ObjectFolder 2.0, a large-scale, multisensory dataset of
common household objects in the form of implicit neural representations that
significantly enhances ObjectFolder 1.0 in three aspects. First, our dataset is
10 times larger in the amount of objects and orders of magnitude faster in
rendering time. Second, we significantly improve the multisensory rendering
quality for all three modalities. Third, we show that models learned from
virtual objects in our dataset successfully transfer to their real-world
counterparts in three challenging tasks: object scale estimation, contact
localization, and shape reconstruction. ObjectFolder 2.0 offers a new path and
testbed for multisensory learning in computer vision and robotics. The dataset
is available at https://github.com/rhgao/ObjectFolder.
