Juvenile state hypothesis: What we can learn from lottery ticket
  hypothesis researches?
  The proposition of lottery ticket hypothesis revealed the relationship
between network structure and initialization parameters and the learning
potential of neural networks. The original lottery ticket hypothesis performs
pruning and weight resetting after training convergence, exposing it to the
problem of forgotten learning knowledge and potential high cost of training.
Therefore, we propose a strategy that combines the idea of neural network
structure search with a pruning algorithm to alleviate this problem. This
algorithm searches and extends the network structure on existing winning ticket
sub-network to producing new winning ticket recursively. This allows the
training and pruning process to continue without compromising performance. A
new winning ticket sub-network with deeper network structure, better
generalization ability and better test performance can be obtained in this
recursive manner. This method can solve: the difficulty of training or
performance degradation of the sub-networks after pruning, the forgetting of
the weights of the original lottery ticket hypothesis and the difficulty of
generating winning ticket sub-network when the final network structure is not
given. We validate this strategy on the MNIST and CIFAR-10 datasets. And after
relating it to similar biological phenomena and relevant lottery ticket
hypothesis studies in recent years, we will further propose a new hypothesis to
discuss which factors that can keep a network juvenile, i.e., those possible
factors that influence the learning potential or generalization performance of
a neural network during training.
