ConceptEvo: Interpreting Concept Evolution in Deep Learning Training
  Deep neural networks (DNNs) have been widely used for decision making,
prompting a surge of interest in interpreting how these complex models work.
Recent literature on DNN interpretation has revolved around already-trained
models; however, much less research focuses on interpreting how the models
evolve as they are trained. Interpreting model evolution is crucial to monitor
network training and can aid proactive decisions about necessary interventions.
In this work, we present ConceptEvo, a general interpretation framework for
DNNs that reveals the inception and evolution of detected concepts during
training. Through a large-scale human evaluation with 260 participants and
quantitative experiments, we show that ConceptEvo discovers evolution across
different models that are meaningful to humans, helpful for early-training
intervention decisions, and crucial to the prediction for a given class.
