Deep Bayesian Self-Training
  Supervised Deep Learning has been highly successful in recent years,
achieving state-of-the-art results in most tasks. However, with the ongoing
uptake of such methods in industrial applications, the requirement for large
amounts of annotated data is often a challenge. In most real world problems,
manual annotation is practically intractable due to time/labour constraints,
thus the development of automated and adaptive data annotation systems is
highly sought after. In this paper, we propose both a (i) Deep Bayesian
Self-Training methodology for automatic data annotation, by leveraging
predictive uncertainty estimates using variational inference and modern Neural
Network architectures, as well as (ii) a practical adaptation procedure for
handling high label variability between different dataset distributions through
clustering of Neural Network latent variable representations. An experimental
study on both public and private datasets is presented illustrating the
superior performance of the proposed approach over standard Self-Training
baselines, highlighting the importance of predictive uncertainty estimates in
safety-critical domains.
