FormulaZero: Distributionally Robust Online Adaptation via Offline
  Population Synthesis
  Balancing performance and safety is crucial to deploying autonomous vehicles
in multi-agent environments. In particular, autonomous racing is a domain that
penalizes safe but conservative policies, highlighting the need for robust,
adaptive strategies. Current approaches either make simplifying assumptions
about other agents or lack robust mechanisms for online adaptation. This work
makes algorithmic contributions to both challenges. First, to generate a
realistic, diverse set of opponents, we develop a novel method for self-play
based on replica-exchange Markov chain Monte Carlo. Second, we propose a
distributionally robust bandit optimization procedure that adaptively adjusts
risk aversion relative to uncertainty in beliefs about opponents' behaviors. We
rigorously quantify the tradeoffs in performance and robustness when
approximating these computations in real-time motion-planning, and we
demonstrate our methods experimentally on autonomous vehicles that achieve
scaled speeds comparable to Formula One racecars.
