MC-LSTM: Mass-Conserving LSTM
  The success of Convolutional Neural Networks (CNNs) in computer vision is
mainly driven by their strong inductive bias, which is strong enough to allow
CNNs to solve vision-related tasks with random weights, meaning without
learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bias
towards storing information over time. However, many real-world systems are
governed by conservation laws, which lead to the redistribution of particular
quantities -- e.g. in physical and economical systems. Our novel
Mass-Conserving LSTM (MC-LSTM) adheres to these conservation laws by extending
the inductive bias of LSTM to model the redistribution of those stored
quantities. MC-LSTMs set a new state-of-the-art for neural arithmetic units at
learning arithmetic operations, such as addition tasks, which have a strong
conservation law, as the sum is constant over time. Further, MC-LSTM is applied
to traffic forecasting, modelling a pendulum, and a large benchmark dataset in
hydrology, where it sets a new state-of-the-art for predicting peak flows. In
the hydrology example, we show that MC-LSTM states correlate with real-world
processes and are therefore interpretable.
