Iterative Pre-Conditioning to Expedite the Gradient-Descent Method
  This paper considers the problem of multi-agent distributed optimization. In
this problem, there are multiple agents in the system, and each agent only
knows its local cost function. The objective for the agents is to collectively
compute a common minimum of the aggregate of all their local cost functions. In
principle, this problem is solvable using a distributed variant of the
traditional gradient-descent method, which is an iterative method. However, the
speed of convergence of the traditional gradient-descent method is highly
influenced by the conditioning of the optimization problem being solved.
Specifically, the method requires a large number of iterations to converge to a
solution if the optimization problem is ill-conditioned.
  In this paper, we propose an iterative pre-conditioning approach that can
significantly attenuate the influence of the problem's conditioning on the
convergence-speed of the gradient-descent method. The proposed pre-conditioning
approach can be easily implemented in distributed systems and has minimal
computation and communication overhead. For now, we only consider a specific
distributed optimization problem wherein the individual local cost functions of
the agents are quadratic. Besides the theoretical guarantees, the improved
convergence speed of our approach is demonstrated through experiments on a real
data-set.
