Distributed Networked Learning with Correlated Data
  We consider a distributed estimation method in a setting with heterogeneous
streams of correlated data distributed across nodes in a network. In the
considered approach, linear models are estimated locally (i.e., with only local
data) subject to a network regularization term that penalizes a local model
that differs from neighboring models. We analyze computation dynamics
(associated with stochastic gradient updates) and information exchange
(associated with exchanging current models with neighboring nodes). We provide
a finite-time characterization of convergence of the weighted ensemble average
estimate and compare this result to federated learning, an alternative approach
to estimation wherein a single model is updated by locally generated gradient
updates. This comparison highlights the trade-off between speed vs precision:
while model updates take place at a faster rate in federated learning, the
proposed networked approach to estimation enables the identification of models
with higher precision. We illustrate the method's general applicability in two
examples: estimating a Markov random field using wireless sensor networks and
modeling prey escape behavior of flocking birds based on a publicly available
dataset.
