Towards Adaptive Training of Agent-based Sparring Partners for Fighter
  Pilots
  A key requirement for the current generation of artificial decision-makers is
that they should adapt well to changes in unexpected situations. This paper
addresses the situation in which an AI for aerial dog fighting, with tunable
parameters that govern its behavior, must optimize behavior with respect to an
objective function that is evaluated and learned through simulations. Bayesian
optimization with a Gaussian Process surrogate is used as the method for
investigating the objective function. One key benefit is that during
optimization, the Gaussian Process learns a global estimate of the true
objective function, with predicted outcomes and a statistical measure of
confidence in areas that haven't been investigated yet. Having a model of the
objective function is important for being able to understand possible outcomes
in the decision space; for example this is crucial for training and providing
feedback to human pilots. However, standard Bayesian optimization does not
perform consistently or provide an accurate Gaussian Process surrogate function
for highly volatile objective functions. We treat these problems by introducing
a novel sampling technique called Hybrid Repeat/Multi-point Sampling. This
technique gives the AI ability to learn optimum behaviors in a highly uncertain
environment. More importantly, it not only improves the reliability of the
optimization, but also creates a better model of the entire objective surface.
With this improved model the agent is equipped to more accurately/efficiently
predict performance in unexplored scenarios.
