Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML
  Systems
  Trade-offs between accuracy and efficiency pervade law, public health, and
other non-computing domains, which have developed policies to guide how to
balance the two in conditions of uncertainty. While computer science also
commonly studies accuracy-efficiency trade-offs, their policy implications
remain poorly examined. Drawing on risk assessment practices in the US, we
argue that, since examining these trade-offs has been useful for guiding
governance in other domains, we need to similarly reckon with these trade-offs
in governing computer systems. We focus our analysis on distributed machine
learning systems. Understanding the policy implications in this area is
particularly urgent because such systems, which include autonomous vehicles,
tend to be high-stakes and safety-critical. We 1) describe how the trade-off
takes shape for these systems, 2) highlight gaps between existing US risk
assessment standards and what these systems require to be properly assessed,
and 3) make specific calls to action to facilitate accountability when
hypothetical risks concerning the accuracy-efficiency trade-off become realized
as accidents in the real world. We close by discussing how such accountability
mechanisms encourage more just, transparent governance aligned with public
values.
