Self-Attentive Ensemble Transformer: Representing Ensemble Interactions
  in Neural Networks for Earth System Models
  Ensemble data from Earth system models has to be calibrated and
post-processed. I propose a novel member-by-member post-processing approach
with neural networks. I bridge ideas from ensemble data assimilation with
self-attention, resulting into the self-attentive ensemble transformer. Here,
interactions between ensemble members are represented as additive and dynamic
self-attentive part. As proof-of-concept, I regress global ECMWF ensemble
forecasts to 2-metre-temperature fields from the ERA5 reanalysis. I demonstrate
that the ensemble transformer can calibrate the ensemble spread and extract
additional information from the ensemble. As it is a member-by-member approach,
the ensemble transformer directly outputs multivariate and spatially-coherent
ensemble members. Therefore, self-attention and the transformer technique can
be a missing piece for a non-parametric post-processing of ensemble data with
neural networks.
