B-SMALL: A Bayesian Neural Network approach to Sparse Model-Agnostic
  Meta-Learning
  There is a growing interest in the learning-to-learn paradigm, also known as
meta-learning, where models infer on new tasks using a few training examples.
Recently, meta-learning based methods have been widely used in few-shot
classification, regression, reinforcement learning, and domain adaptation. The
model-agnostic meta-learning (MAML) algorithm is a well-known algorithm that
obtains model parameter initialization at meta-training phase. In the meta-test
phase, this initialization is rapidly adapted to new tasks by using gradient
descent. However, meta-learning models are prone to overfitting since there are
insufficient training tasks resulting in over-parameterized models with poor
generalization performance for unseen tasks. In this paper, we propose a
Bayesian neural network based MAML algorithm, which we refer to as the B-SMALL
algorithm. The proposed framework incorporates a sparse variational loss term
alongside the loss function of MAML, which uses a sparsifying approximated KL
divergence as a regularizer. We demonstrate the performance of B-MAML using
classification and regression tasks, and highlight that training a sparsifying
BNN using MAML indeed improves the parameter footprint of the model while
performing at par or even outperforming the MAML approach. We also illustrate
applicability of our approach in distributed sensor networks, where sparsity
and meta-learning can be beneficial.
