Looper: An end-to-end ML platform for product decisions
  Modern software systems and products increasingly rely on machine learning
models to make data-driven decisions based on interactions with users,
infrastructure and other systems. For broader adoption, this practice must (i)
accommodate product engineers without ML backgrounds, (ii) support finegrain
product-metric evaluation and (iii) optimize for product goals. To address
shortcomings of prior platforms, we introduce general principles for and the
architecture of an ML platform, Looper, with simple APIs for decision-making
and feedback collection. Looper covers the end-to-end ML lifecycle from
collecting training data and model training to deployment and inference, and
extends support to personalization, causal evaluation with heterogenous
treatment effects, and Bayesian tuning for product goals. During the 2021
production deployment Looper simultaneously hosted 440-1,000 ML models that
made 4-6 million real-time decisions per second. We sum up experiences of
platform adopters and describe their learning curve.
