Efficient and Robust Machine Learning for Real-World Systems
  While machine learning is traditionally a resource intensive task, embedded
systems, autonomous navigation and the vision of the Internet-of-Things fuel
the interest in resource efficient approaches. These approaches require a
carefully chosen trade-off between performance and resource consumption in
terms of computation and energy. On top of this, it is crucial to treat
uncertainty in a consistent manner in all but the simplest applications of
machine learning systems. In particular, a desideratum for any real-world
system is to be robust in the presence of outliers and corrupted data, as well
as being `aware' of its limits, i.e.\ the system should maintain and provide an
uncertainty estimate over its own predictions. These complex demands are among
the major challenges in current machine learning research and key to ensure a
smooth transition of machine learning technology into every day's applications.
In this article, we provide an overview of the current state of the art of
machine learning techniques facilitating these real-world requirements. First
we provide a comprehensive review of resource-efficiency in deep neural
networks with focus on techniques for model size reduction, compression and
reduced precision. These techniques can be applied during training or as
post-processing and are widely used to reduce both computational complexity and
memory footprint. As most (practical) neural networks are limited in their ways
to treat uncertainty, we contrast them with probabilistic graphical models,
which readily serve these desiderata by means of probabilistic inference. In
that way, we provide an extensive overview of the current state-of-the-art of
robust and efficient machine learning for real-world systems.
