A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air
  Traffic Control
  Air traffic control is an example of a highly challenging operational problem
that is readily amenable to human expertise augmentation via decision support
technologies. In this paper, we propose a new intelligent decision making
framework that leverages multi-agent reinforcement learning (MARL) to
dynamically suggest adjustments of aircraft speeds in real-time. The goal of
the system is to enhance the ability of an air traffic controller to provide
effective guidance to aircraft to avoid air traffic congestion, near-miss
situations, and to improve arrival timeliness. We develop a novel deep ensemble
MARL method that can concisely capture the complexity of the air traffic
control problem by learning to efficiently arbitrate between the decisions of a
local kernel-based RL model and a wider-reaching deep MARL model. The proposed
method is trained and evaluated on an open-source air traffic management
simulator developed by Eurocontrol. Extensive empirical results on a real-world
dataset including thousands of aircraft demonstrate the feasibility of using
multi-agent RL for the problem of en-route air traffic control and show that
our proposed deep ensemble MARL method significantly outperforms three
state-of-the-art benchmark approaches.
