Multi-Loss Sub-Ensembles for Accurate Classification with Uncertainty
  Estimation
  Deep neural networks (DNNs) have made a revolution in numerous fields during
the last decade. However, in tasks with high safety requirements, such as
medical or autonomous driving applications, providing an assessment of the
models reliability can be vital. Uncertainty estimation for DNNs has been
addressed using Bayesian methods, providing mathematically founded models for
reliability assessment. These model are computationally expensive and generally
impractical for many real-time use cases. Recently, non-Bayesian methods were
proposed to tackle uncertainty estimation more efficiently. We propose an
efficient method for uncertainty estimation in DNNs achieving high accuracy. We
simulate the notion of multi-task learning on single-task problems by producing
parallel predictions from similar models differing by their loss. This
multi-loss approach allows one-phase training for single-task learning with
uncertainty estimation. We keep our inference time relatively low by leveraging
the advantage proposed by the Deep-Sub-Ensembles method. The novelty of this
work resides in the proposed accurate variational inference with a simple and
convenient training procedure, while remaining competitive in terms of
computational time. We conduct experiments on SVHN, CIFAR10, CIFAR100 as well
as Image-Net using different architectures. Our results show improved accuracy
on the classification task and competitive results on several uncertainty
measures.
