Complexity of Stochastic Dual Dynamic Programming
  Stochastic dual dynamic programming is a cutting plane type algorithm for
multi-stage stochastic optimization originated about 30 years ago. In spite of
its popularity in practice, there does not exist any analysis on the
convergence rates of this method. In this paper, we first establish the number
of iterations, i.e., iteration complexity, required by a basic dynamic cutting
plane method for solving relatively simple multi-stage optimization problems,
by introducing novel mathematical tools including the saturation of search
points. We then refine these basic tools and establish the iteration complexity
for both deterministic and stochastic dual dynamic programming methods for
solving more general multi-stage stochastic optimization problems under the
standard stage-wise independence assumption. Our results indicate that the
complexity of some deterministic variants of these methods mildly increases
with the number of stages $T$, in fact linearly dependent on $T$ for discounted
problems. Therefore, they are efficient for strategic decision making which
involves a large number of stages, but with a relatively small number of
decision variables in each stage. Without explicitly discretizing the state and
action spaces, these methods might also be pertinent to the related
reinforcement learning and stochastic control areas.
