Domain-Adversarial and Conditional State Space Model for Imitation
  Learning
  State representation learning (SRL) in partially observable Markov decision
processes has been studied to learn abstract features of data useful for robot
control tasks. For SRL, acquiring domain-agnostic states is essential for
achieving efficient imitation learning. Without these states, imitation
learning is hampered by domain-dependent information useless for control.
However, existing methods fail to remove such disturbances from the states when
the data from experts and agents show large domain shifts. To overcome this
issue, we propose a domain-adversarial and conditional state space model
(DAC-SSM) that enables control systems to obtain domain-agnostic and task- and
dynamics-aware states. DAC-SSM jointly optimizes the state inference,
observation reconstruction, forward dynamics, and reward models. To remove
domain-dependent information from the states, the model is trained with domain
discriminators in an adversarial manner, and the reconstruction is conditioned
on domain labels. We experimentally evaluated the model predictive control
performance via imitation learning for continuous control of sparse reward
tasks in simulators and compared it with the performance of the existing SRL
method. The agents from DAC-SSM achieved performance comparable to experts and
more than twice the baselines. We conclude domain-agnostic states are essential
for imitation learning that has large domain shifts and can be obtained using
DAC-SSM.
