Age-Based Coded Computation for Bias Reduction in Distributed Learning
  Coded computation can be used to speed up distributed learning in the
presence of straggling workers. Partial recovery of the gradient vector can
further reduce the computation time at each iteration; however, this can result
in biased estimators, which may slow down convergence, or even cause
divergence. Estimator bias will be particularly prevalent when the straggling
behavior is correlated over time, which results in the gradient estimators
being dominated by a few fast servers. To mitigate biased estimators, we design
a $timely$ dynamic encoding framework for partial recovery that includes an
ordering operator that changes the codewords and computation orders at workers
over time. To regulate the recovery frequencies, we adopt an $age$ metric in
the design of the dynamic encoding scheme. We show through numerical results
that the proposed dynamic encoding strategy increases the timeliness of the
recovered computations, which as a result, reduces the bias in model updates,
and accelerates the convergence compared to the conventional static partial
recovery schemes.
