Be Like Water: Robustness to Extraneous Variables Via Adaptive Feature
  Normalization
  Extraneous variables are variables that are irrelevant for a certain task,
but heavily affect the distribution of the available data. In this work, we
show that the presence of such variables can degrade the performance of
deep-learning models. We study three datasets where there is a strong influence
of known extraneous variables: classification of upper-body movements in stroke
patients, annotation of surgical activities, and recognition of corrupted
images. Models trained with batch normalization learn features that are highly
dependent on the extraneous variables. In batch normalization, the statistics
used to normalize the features are learned from the training set and fixed at
test time, which produces a mismatch in the presence of varying extraneous
variables. We demonstrate that estimating the feature statistics adaptively
during inference, as in instance normalization, addresses this issue, producing
normalized features that are more robust to changes in the extraneous
variables. This results in a significant gain in performance for different
network architectures and choices of feature statistics.
