Growing Representation Learning
  Machine learning continues to grow in popularity due to its ability to learn
increasingly complex tasks. However, for many supervised models, the shift in a
data distribution or the appearance of a new event can result in a severe
decrease in model performance. Retraining a model from scratch with updated
data can be resource intensive or impossible depending on the constraints
placed on an organization or system. Continual learning methods attempt to
adapt models to new classes instead of retraining. However, many of these
methods do not have a detection method for new classes or make assumptions
about the distribution of classes. In this paper, we develop an attention based
Gaussian Mixture, called GMAT, that learns interpretable representations of
data with or without labels. We incorporate this method with existing Neural
Architecture Search techniques to develop an algorithm for detection new events
for an optimal number of representations through an iterative process of
training a growing. We show that our method is capable learning new
representations of data without labels or assumptions about the distributions
of labels. We additionally develop a method that allows our model to utilize
labels to more accurately develop representations. Lastly, we show that our
method can avoid catastrophic forgetting by replaying samples from learned
representations.
