Metamorphic Testing of a Deep Learning based Forecaster
  In this paper, we present the Metamorphic Testing of an in-use deep learning
based forecasting application. The application looks at the past data of system
characteristics (e.g. `memory allocation') to predict outages in the future. We
focus on two statistical / machine learning based components - a) detection of
co-relation between system characteristics and b) estimating the future value
of a system characteristic using an LSTM (a deep learning architecture). In
total, 19 Metamorphic Relations have been developed and we provide proofs &
algorithms where applicable. We evaluated our method through two settings. In
the first, we executed the relations on the actual application and uncovered 8
issues not known before. Second, we generated hypothetical bugs, through
Mutation Testing, on a reference implementation of the LSTM based forecaster
and found that 65.9% of the bugs were caught through the relations.
