Hierarchical Indian Buffet Neural Networks for Bayesian Continual
  Learning
  We place an Indian Buffet process (IBP) prior over the structure of a
Bayesian Neural Network (BNN), thus allowing the complexity of the BNN to
increase and decrease automatically. We further extend this model such that the
prior on the structure of each hidden layer is shared globally across all
layers, using a Hierarchical-IBP (H-IBP). We apply this model to the problem of
resource allocation in Continual Learning (CL) where new tasks occur and the
network requires extra resources. Our model uses online variational inference
with reparameterisation of the Bernoulli and Beta distributions, which
constitute the IBP and H-IBP priors. As we automatically learn the number of
weights in each layer of the BNN, overfitting and underfitting problems are
largely overcome. We show empirically that our approach offers a competitive
edge over existing methods in CL.
