Slice-based Learning: A Programming Model for Residual Learning in
  Critical Data Slices
  In real-world machine learning applications, data subsets correspond to
especially critical outcomes: vulnerable cyclist detections are safety-critical
in an autonomous driving task, and "question" sentences might be important to a
dialogue agent's language understanding for product purposes. While machine
learning models can achieve high quality performance on coarse-grained metrics
like F1-score and overall accuracy, they may underperform on critical
subsets---we define these as slices, the key abstraction in our approach. To
address slice-level performance, practitioners often train separate "expert"
models on slice subsets or use multi-task hard parameter sharing. We propose
Slice-based Learning, a new programming model in which the slicing function
(SF), a programming interface, specifies critical data subsets for which the
model should commit additional capacity. Any model can leverage SFs to learn
slice expert representations, which are combined with an attention mechanism to
make slice-aware predictions. We show that our approach maintains a
parameter-efficient representation while improving over baselines by up to 19.0
F1 on slices and 4.6 F1 overall on datasets spanning language understanding
(e.g. SuperGLUE), computer vision, and production-scale industrial systems.
