A Note on Learning Rare Events in Molecular Dynamics using LSTM and
  Transformer
  Recurrent neural networks for language models like long short-term memory
(LSTM) have been utilized as a tool for modeling and predicting long term
dynamics of complex stochastic molecular systems. Recently successful examples
on learning slow dynamics by LSTM are given with simulation data of low
dimensional reaction coordinate. However, in this report we show that the
following three key factors significantly affect the performance of language
model learning, namely dimensionality of reaction coordinates, temporal
resolution and state partition. When applying recurrent neural networks to
molecular dynamics simulation trajectories of high dimensionality, we find that
rare events corresponding to the slow dynamics might be obscured by other
faster dynamics of the system, and cannot be efficiently learned. Under such
conditions, we find that coarse graining the conformational space into
metastable states and removing recrossing events when estimating transition
probabilities between states could greatly help improve the accuracy of slow
dynamics learning in molecular dynamics. Moreover, we also explore other models
like Transformer, which do not show superior performance than LSTM in
overcoming these issues. Therefore, to learn rare events of slow molecular
dynamics by LSTM and Transformer, it is critical to choose proper temporal
resolution (i.e., saving intervals of MD simulation trajectories) and state
partition in high resolution data, since deep neural network models might not
automatically disentangle slow dynamics from fast dynamics when both are
present in data influencing each other.
