Optimal Cycling of a Heterogenous Battery Bank via Reinforcement
  Learning
  We consider the problem of optimal charging/discharging of a bank of
heterogenous battery units, driven by stochastic electricity generation and
demand processes. The batteries in the battery bank may differ with respect to
their capacities, ramp constraints, losses, as well as cycling costs. The goal
is to minimize the degradation costs associated with battery cycling in the
long run; this is posed formally as a Markov decision process. We propose a
linear function approximation based Q-learning algorithm for learning the
optimal solution, using a specially designed class of kernel functions that
approximate the structure of the value functions associated with the MDP. The
proposed algorithm is validated via an extensive case study.
