Coded Computation across Shared Heterogeneous Workers with Communication
  Delay
  Distributed computing enables large-scale computation tasks to be processed
over multiple workers in parallel. However, the randomness of communication and
computation delays across workers causes the straggler effect, which may
degrade the performance. Coded computation helps to mitigate the straggler
effect, but the amount of redundant load and their assignment to the workers
should be carefully optimized. In this work, we consider a multi-master
heterogeneous-worker distributed computing scenario, where multiple matrix
multiplication tasks are encoded and allocated to workers for parallel
computation. The goal is to minimize the communication plus computation delay
of the slowest task. We propose worker assignment, resource allocation and load
allocation algorithms under both dedicated and fractional worker assignment
policies, where each worker can process the encoded tasks of either a single
master or multiple masters, respectively. Then, the non-convex delay
minimization problem is solved by employing the Markov's inequality-based
approximation, Karush-Kuhn-Tucker conditions, and successive convex
approximation methods. Through extensive simulations, we show that the proposed
algorithms can reduce the task completion delay compared to the benchmarks, and
observe that dedicated and fractional worker assignment policies have different
scopes of applications.
