Research on Autonomous Maneuvering Decision of UCAV based on Approximate
  Dynamic Programming
  Unmanned aircraft systems can perform some more dangerous and difficult
missions than manned aircraft systems. In some highly complicated and
changeable tasks, such as air combat, the maneuvering decision mechanism is
required to sense the combat situation accurately and make the optimal strategy
in real-time. This paper presents a formulation of a 3-D one-on-one air combat
maneuvering problem and an approximate dynamic programming approach for
computing an optimal policy on autonomous maneuvering decision making. The
aircraft learns combat strategies in a Reinforcement Leaning method, while
sensing the environment, taking available maneuvering actions and getting
feedback reward signals. To solve the problem of dimensional explosion in the
air combat, the proposed method is implemented through feature selection,
trajectory sampling, function approximation and Bellman backup operation in the
air combat simulation environment. This approximate dynamic programming
approach provides a fast response to a rapidly changing tactical situation,
learns in long-term planning, without any explicitly coded air combat rule
base.
