Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win
  the Jackpot?
  There have been long-standing controversies and inconsistencies over the
experiment setup and criteria for identifying the "winning ticket" in
literature. To reconcile such, we revisit the definition of lottery ticket
hypothesis, with comprehensive and more rigorous conditions. Under our new
definition, we show concrete evidence to clarify whether the winning ticket
exists across the major DNN architectures and/or applications. Through
extensive experiments, we perform quantitative analysis on the correlations
between winning tickets and various experimental factors, and empirically study
the patterns of our observations. We find that the key training
hyperparameters, such as learning rate and training epochs, as well as the
architecture characteristics such as capacities and residual connections, are
all highly correlated with whether and when the winning tickets can be
identified. Based on our analysis, we summarize a guideline for parameter
settings in regards of specific architecture characteristics, which we hope to
catalyze the research progress on the topic of lottery ticket hypothesis. Our
codes are publicly available at:
https://github.com/boone891214/sanity-check-LTH.
