Robust Neural Regression via Uncertainty Learning
  Deep neural networks tend to underestimate uncertainty and produce overly
confident predictions. Recently proposed solutions, such as MC Dropout and
SDENet, require complex training and/or auxiliary out-of-distribution data. We
propose a simple solution by extending the time-tested iterative reweighted
least square (IRLS) in generalised linear regression. We use two sub-networks
to parametrise the prediction and uncertainty estimation, enabling easy
handling of complex inputs and nonlinear response. The two sub-networks have
shared representations and are trained via two complementary loss functions for
the prediction and the uncertainty estimates, with interleaving steps as in a
cooperative game. Compared with more complex models such as MC-Dropout or
SDE-Net, our proposed network is simpler to implement and more robust
(insensitive to varying aleatoric and epistemic uncertainty).
