LeanML: A Design Pattern To Slash Avoidable Wastes in Machine Learning
  Projects
  We introduce the first application of the lean methodology to machine
learning projects. Similar to lean startups and lean manufacturing, we argue
that lean machine learning (LeanML) can drastically slash avoidable wastes in
commercial machine learning projects, reduce the business risk in investing in
machine learning capabilities and, in so doing, further democratize access to
machine learning. The lean design pattern we propose in this paper is based on
two realizations. First, it is possible to estimate the best performance one
may achieve when predicting an outcome $y \in \mathcal{Y}$ using a given set of
explanatory variables $x \in \mathcal{X}$, for a wide range of performance
metrics, and without training any predictive model. Second, doing so is
considerably easier, faster, and cheaper than learning the best predictive
model. We derive formulae expressing the best $R^2$, MSE, classification
accuracy, and log-likelihood per observation achievable when using $x$ to
predict $y$ as a function of the mutual information $I\left(y; x\right)$, and
possibly a measure of the variability of $y$ (e.g. its Shannon entropy in the
case of classification accuracy, and its variance in the case regression MSE).
We illustrate the efficacy of the LeanML design pattern on a wide range of
regression and classification problems, synthetic and real-life.
