A Note on High-Probability versus In-Expectation Guarantees of
  Generalization Bounds in Machine Learning
  Statistical machine learning theory often tries to give generalization
guarantees of machine learning models. Those models naturally underlie some
fluctuation, as they are based on a data sample. If we were unlucky, and
gathered a sample that is not representative of the underlying distribution,
one cannot expect to construct a reliable machine learning model. Following
that, statements made about the performance of machine learning models have to
take the sampling process into account. The two common approaches for that are
to generate statements that hold either in high-probability, or in-expectation,
over the random sampling process. In this short note we show how one may
transform one statement to another. As a technical novelty we address the case
of unbounded loss function, where we use a fairly new assumption, called the
witness condition.
