Hierarchical Reinforcement Learning for Air-to-Air Combat
  Artificial Intelligence (AI) is becoming a critical component in the defense
industry, as recently demonstrated by DARPA`s AlphaDogfight Trials (ADT). ADT
sought to vet the feasibility of AI algorithms capable of piloting an F-16 in
simulated air-to-air combat. As a participant in ADT, Lockheed Martin`s (LM)
approach combines a hierarchical architecture with maximum-entropy
reinforcement learning (RL), integrates expert knowledge through reward
shaping, and supports modularity of policies. This approach achieved a $2^{nd}$
place finish in the final ADT event (among eight total competitors) and
defeated a graduate of the US Air Force's (USAF) F-16 Weapons Instructor Course
in match play.
