Feature-Attending Recurrent Modules for Generalizing Object-Centric
  Behavior
  To generalize in object-centric tasks, a reinforcement learning (RL) agent
needs to exploit the structure that objects induce. Prior work has either
hard-coded object-centric features, used complex object-centric generative
models, or updated state using local spatial features. However, these
approaches have had limited success in enabling general RL agents. Motivated by
this, we introduce ``Feature-Attending Recurrent Modules'' (FARM), an
architecture for learning state representations that relies on simple, broadly
applicable inductive biases for capturing spatial and temporal regularities.
FARM learns a state representation that is distributed across multiple modules
that each attend to spatiotemporal features with an expressive feature
attention mechanism. This enables FARM to represent diverse object-induced
spatial and temporal regularities across subsets of modules. We hypothesize
that this enables an RL agent to flexibly recombine its experiences for
generalization. We study task suites in both 2D and 3D environments and find
that FARM better generalizes compared to competing architectures that leverage
attention or multiple modules.
