Exploring Complementary Strengths of Invariant and Equivariant
  Representations for Few-Shot Learning
  In many real-world problems, collecting a large number of labeled samples is
infeasible. Few-shot learning (FSL) is the dominant approach to address this
issue, where the objective is to quickly adapt to novel categories in presence
of a limited number of samples. FSL tasks have been predominantly solved by
leveraging the ideas from gradient-based meta-learning and metric learning
approaches. However, recent works have demonstrated the significance of
powerful feature representations with a simple embedding network that can
outperform existing sophisticated FSL algorithms. In this work, we build on
this insight and propose a novel training mechanism that simultaneously
enforces equivariance and invariance to a general set of geometric
transformations. Equivariance or invariance has been employed standalone in the
previous works; however, to the best of our knowledge, they have not been used
jointly. Simultaneous optimization for both of these contrasting objectives
allows the model to jointly learn features that are not only independent of the
input transformation but also the features that encode the structure of
geometric transformations. These complementary sets of features help generalize
well to novel classes with only a few data samples. We achieve additional
improvements by incorporating a novel self-supervised distillation objective.
Our extensive experimentation shows that even without knowledge distillation
our proposed method can outperform current state-of-the-art FSL methods on five
popular benchmark datasets.
