Distributed Algorithms for Linearly-Solvable Optimal Control in
  Networked Multi-Agent Systems
  Distributed algorithms for both discrete-time and continuous-time linearly
solvable optimal control (LSOC) problems of networked multi-agent systems
(MASs) are investigated in this paper. A distributed framework is proposed to
partition the optimal control problem of a networked MAS into several local
optimal control problems in factorial subsystems, such that each (central)
agent behaves optimally to minimize the joint cost function of a subsystem that
comprises a central agent and its neighboring agents, and the local control
actions (policies) only rely on the knowledge of local observations. Under this
framework, we not only preserve the correlations between neighboring agents,
but moderate the communication and computational complexities by decentralizing
the sampling and computational processes over the network. For discrete-time
systems modeled by Markov decision processes, the joint Bellman equation of
each subsystem is transformed into a system of linear equations and solved
using parallel programming. For continuous-time systems modeled by It\^o
diffusion processes, the joint optimality equation of each subsystem is
converted into a linear partial differential equation, whose solution is
approximated by a path integral formulation and a sample-efficient relative
entropy policy search algorithm, respectively. The learned control policies are
generalized to solve the unlearned tasks by resorting to the compositionality
principle, and illustrative examples of cooperative UAV teams are provided to
verify the effectiveness and advantages of these algorithms.
