Garfield: System Support for Byzantine Machine Learning
  We present Garfield, a library to transparently make machine learning (ML)
applications, initially built with popular (but fragile) frameworks, e.g.,
TensorFlow and PyTorch, Byzantine-resilient. Garfield relies on a novel
object-oriented design, reducing the coding effort, and addressing the
vulnerability of the shared-graph architecture followed by classical ML
frameworks. Garfield encompasses various communication patterns and supports
computations on CPUs and GPUs, allowing addressing the general question of the
very practical cost of Byzantine resilience in SGD-based ML applications. We
report on the usage of Garfield on three main ML architectures: (a) a single
server with multiple workers, (b) several servers and workers, and (c)
peer-to-peer settings. Using Garfield, we highlight several interesting facts
about the cost of Byzantine resilience. In particular, (a) Byzantine
resilience, unlike crash resilience, induces an accuracy loss, (b) the
throughput overhead comes more from communication than from robust aggregation,
and (c) tolerating Byzantine servers costs more than tolerating Byzantine
workers.
