AtmoDist: Self-supervised Representation Learning for Atmospheric
  Dynamics
  Representation learning has proven to be a powerful methodology in a wide
variety of machine learning applications. For atmospheric dynamics, however, it
has so far not been considered, arguably due to the lack of large-scale,
labeled datasets that could be used for training. In this work, we show that
the difficulty is benign and introduce a self-supervised learning task that
defines a categorical loss for a wide variety of unlabeled atmospheric
datasets. Specifically, we train a neural network on the simple yet intricate
task of predicting the temporal distance between atmospheric fields, e.g. the
components of the wind field, from distinct but nearby times. Despite this
simplicity, a neural network will provide good predictions only when it
develops an internal representation that captures intrinsic aspects of
atmospheric dynamics. We demonstrate this by introducing a data-driven distance
metric for atmospheric states based on representations learned from ERA5
reanalysis. When employ as a loss function for downscaling, this Atmodist
distance leads to downscaled fields that match the true statistics more closely
than the previous state-of-the-art based on an l2-loss and whose local behavior
is more realistic. Since it is derived from observational data, AtmoDist also
provides a novel perspective on atmospheric predictability.
