Learning In Practice: Reasoning About Quantization
  There is a mismatch between the standard theoretical analyses of statistical
machine learning and how learning is used in practice. The foundational
assumption supporting the theory is that we can represent features and models
using real-valued parameters. In practice, however, we do not use real numbers
at any point during training or deployment. Instead, we rely on discrete and
finite quantizations of the reals, typically floating points. In this paper, we
propose a framework for reasoning about learning under arbitrary quantizations.
Using this formalization, we prove the convergence of quantization-aware
versions of the Perceptron and Frank-Wolfe algorithms. Finally, we report the
results of an extensive empirical study of the impact of quantization using a
broad spectrum of datasets.
