Learngene: From Open-World to Your Learning Task
  Although deep learning has made significant progress on fixed large-scale
datasets, it typically encounters challenges regarding improperly detecting
unknown/unseen classes in the open-world scenario, over-parametrized, and
overfitting small samples. Since biological systems can overcome the above
difficulties very well, individuals inherit an innate gene from collective
creatures that have evolved over hundreds of millions of years and then learn
new skills through few examples. Inspired by this, we propose a practical
collective-individual paradigm where an evolution (expandable) network is
trained on sequential tasks and then recognize unknown classes in real-world.
Moreover, the learngene, i.e., the gene for learning initialization rules of
the target model, is proposed to inherit the meta-knowledge from the collective
model and reconstruct a lightweight individual model on the target task.
Particularly, a novel criterion is proposed to discover learngene in the
collective model, according to the gradient information. Finally, the
individual model is trained only with few samples on the target learning tasks.
We demonstrate the effectiveness of our approach in an extensive empirical
study and theoretical analysis.
