An Alternative Probabilistic Interpretation of the Huber Loss
  The Huber loss is a robust loss function used for a wide range of regression
tasks. To utilize the Huber loss, a parameter that controls the transitions
from a quadratic function to an absolute value function needs to be selected.
We believe the standard probabilistic interpretation that relates the Huber
loss to the Huber density fails to provide adequate intuition for identifying
the transition point. As a result, a hyper-parameter search is often necessary
to determine an appropriate value. In this work, we propose an alternative
probabilistic interpretation of the Huber loss, which relates minimizing the
loss to minimizing an upper-bound on the Kullback-Leibler divergence between
Laplace distributions, where one distribution represents the noise in the
ground-truth and the other represents the noise in the prediction. In addition,
we show that the parameters of the Laplace distributions are directly related
to the transition point of the Huber loss. We demonstrate, through a toy
problem, that the optimal transition point of the Huber loss is closely related
to the distribution of the noise in the ground-truth data. As a result, our
interpretation provides an intuitive way to identify well-suited
hyper-parameters by approximating the amount of noise in the data, which we
demonstrate through a case study and experimentation on the Faster R-CNN and
RetinaNet object detectors.
