Reduced-Dimensional Reinforcement Learning Control using Singular
  Perturbation Approximations
  We present a set of model-free, reduced-dimensional reinforcement learning
(RL) based optimal control designs for linear time-invariant singularly
perturbed (SP) systems. We first present a state-feedback and output-feedback
based RL control design for a generic SP system with unknown state and input
matrices. We take advantage of the underlying time-scale separation property of
the plant to learn a linear quadratic regulator (LQR) for only its slow
dynamics, thereby saving a significant amount of learning time compared to the
conventional full-dimensional RL controller. We analyze the sub-optimality of
the design using SP approximation theorems and provide sufficient conditions
for closed-loop stability. Thereafter, we extend both designs to clustered
multi-agent consensus networks, where the SP property reflects through
clustering. We develop both centralized and cluster-wise block-decentralized RL
controllers for such networks, in reduced dimensions. We demonstrate the
details of the implementation of these controllers using simulations of
relevant numerical examples and compare them with conventional RL designs to
show the computational benefits of our approach.
