Building Generalizable Agents with a Realistic and Rich 3D Environment
  Teaching an agent to navigate in an unseen 3D environment is a challenging
task, even in the event of simulated environments. To generalize to unseen
environments, an agent needs to be robust to low-level variations (e.g. color,
texture, object changes), and also high-level variations (e.g. layout changes
of the environment). To improve overall generalization, all types of variations
in the environment have to be taken under consideration via different level of
data augmentation steps. To this end, we propose House3D, a rich, extensible
and efficient environment that contains 45,622 human-designed 3D scenes of
visually realistic houses, ranging from single-room studios to multi-storied
houses, equipped with a diverse set of fully labeled 3D objects, textures and
scene layouts, based on the SUNCG dataset (Song et.al.). The diversity in
House3D opens the door towards scene-level augmentation, while the label-rich
nature of House3D enables us to inject pixel- & task-level augmentations such
as domain randomization (Toubin et. al.) and multi-task training. Using a
subset of houses in House3D, we show that reinforcement learning agents trained
with an enhancement of different levels of augmentations perform much better in
unseen environments than our baselines with raw RGB input by over 8% in terms
of navigation success rate. House3D is publicly available at
http://github.com/facebookresearch/House3D.
