DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal
  Causality of Deep Classification Training
  Understanding how the predictions of deep learning models are formed during
the training process is crucial to improve model performance and fix model
defects, especially when we need to investigate nontrivial training strategies
such as active learning, and track the root cause of unexpected training
results such as performance degeneration.
  In this work, we propose a time-travelling visual solution DeepVisualInsight
(DVI), aiming to manifest the spatio-temporal causality while training a deep
learning image classifier. The spatio-temporal causality demonstrates how the
gradient-descent algorithm and various training data sampling techniques can
influence and reshape the layout of learnt input representation and the
classification boundaries in consecutive epochs. Such causality allows us to
observe and analyze the whole learning process in the visible low dimensional
space. Technically, we propose four spatial and temporal properties and design
our visualization solution to satisfy them. These properties preserve the most
important information when inverse-)projecting input samples between the
visible low-dimensional and the invisible high-dimensional space, for causal
analyses. Our extensive experiments show that, comparing to baseline
approaches, we achieve the best visualization performance regarding the
spatial/temporal properties and visualization efficiency. Moreover, our case
study shows that our visual solution can well reflect the characteristics of
various training scenarios, showing good potential of DVI as a debugging tool
for analyzing deep learning training processes.
