Learning Deep Neural Networks under Agnostic Corrupted Supervision
  Training deep neural models in the presence of corrupted supervision is
challenging as the corrupted data points may significantly impact the
generalization performance. To alleviate this problem, we present an efficient
robust algorithm that achieves strong guarantees without any assumption on the
type of corruption and provides a unified framework for both classification and
regression problems. Unlike many existing approaches that quantify the quality
of the data points (e.g., based on their individual loss values), and filter
them accordingly, the proposed algorithm focuses on controlling the collective
impact of data points on the average gradient. Even when a corrupted data point
failed to be excluded by our algorithm, the data point will have a very limited
impact on the overall loss, as compared with state-of-the-art filtering methods
based on loss values. Extensive experiments on multiple benchmark datasets have
demonstrated the robustness of our algorithm under different types of
corruption.
