Stability and Performance Limits of Adaptive Primal-Dual Networks
  This work studies distributed primal-dual strategies for adaptation and
learning over networks from streaming data. Two first-order methods are
considered based on the Arrow-Hurwicz (AH) and augmented Lagrangian (AL)
techniques. Several revealing results are discovered in relation to the
performance and stability of these strategies when employed over adaptive
networks. The conclusions establish that the advantages that these methods have
for deterministic optimization problems do not necessarily carry over to
stochastic optimization problems. It is found that they have narrower stability
ranges and worse steady-state mean-square-error performance than primal methods
of the consensus and diffusion type. It is also found that the AH technique can
become unstable under a partial observation model, while the other techniques
are able to recover the unknown under this scenario. A method to enhance the
performance of AL strategies is proposed by tying the selection of the
step-size to their regularization parameter. It is shown that this method
allows the AL algorithm to approach the performance of consensus and diffusion
strategies but that it remains less stable than these other strategies.
