Cloud K-SVD: A Collaborative Dictionary Learning Algorithm for Big,
  Distributed Data
  This paper studies the problem of data-adaptive representations for big,
distributed data. It is assumed that a number of geographically-distributed,
interconnected sites have massive local data and they are interested in
collaboratively learning a low-dimensional geometric structure underlying these
data. In contrast to previous works on subspace-based data representations,
this paper focuses on the geometric structure of a union of subspaces (UoS). In
this regard, it proposes a distributed algorithm---termed cloud K-SVD---for
collaborative learning of a UoS structure underlying distributed data of
interest. The goal of cloud K-SVD is to learn a common overcomplete dictionary
at each individual site such that every sample in the distributed data can be
represented through a small number of atoms of the learned dictionary. Cloud
K-SVD accomplishes this goal without requiring exchange of individual samples
between sites. This makes it suitable for applications where sharing of raw
data is discouraged due to either privacy concerns or large volumes of data.
This paper also provides an analysis of cloud K-SVD that gives insights into
its properties as well as deviations of the dictionaries learned at individual
sites from a centralized solution in terms of different measures of
local/global data and topology of interconnections. Finally, the paper
numerically illustrates the efficacy of cloud K-SVD on real and synthetic
distributed data.
