Model-Based Opponent Modeling
  When one agent interacts with a multi-agent environment, it is challenging to
deal with various opponents unseen before. Modeling the behaviors, goals, or
beliefs of opponents could help the agent adjust its policy to adapt to
different opponents. In addition, it is also important to consider opponents
who are learning simultaneously or capable of reasoning. However, existing work
usually tackles only one of the aforementioned types of opponent. In this
paper, we propose model-based opponent modeling (MBOM), which employs the
environment model to adapt to all kinds of opponent. MBOM simulates the
recursive reasoning process in the environment model and imagines a set of
improving opponent policies. To effectively and accurately represent the
opponent policy, MBOM further mixes the imagined opponent policies according to
the similarity with the real behaviors of opponents. Empirically, we show that
MBOM achieves more effective adaptation than existing methods in competitive
and cooperative environments, respectively with different types of opponent,
i.e., fixed policy, na\"ive learner, and reasoning learner.
