Communication Lower Bounds for Distributed Convex Optimization:
  Partition Data on Features
  Recently, there has been an increasing interest in designing distributed
convex optimization algorithms under the setting where the data matrix is
partitioned on features. Algorithms under this setting sometimes have many
advantages over those under the setting where data is partitioned on samples,
especially when the number of features is huge. Therefore, it is important to
understand the inherent limitations of these optimization problems. In this
paper, with certain restrictions on the communication allowed in the
procedures, we develop tight lower bounds on communication rounds for a broad
class of non-incremental algorithms under this setting. We also provide a lower
bound on communication rounds for a class of (randomized) incremental
algorithms.
