The Two Dimensions of Worst-case Training and the Integrated Effect for
  Out-of-domain Generalization
  Training with an emphasis on "hard-to-learn" components of the data has been
proven as an effective method to improve the generalization of machine learning
models, especially in the settings where robustness (e.g., generalization
across distributions) is valued. Existing literature discussing this
"hard-to-learn" concept are mainly expanded either along the dimension of the
samples or the dimension of the features. In this paper, we aim to introduce a
simple view merging these two dimensions, leading to a new, simple yet
effective, heuristic to train machine learning models by emphasizing the
worst-cases on both the sample and the feature dimensions. We name our method
W2D following the concept of "Worst-case along Two Dimensions". We validate the
idea and demonstrate its empirical strength over standard benchmarks.
