Manas: Mining Software Repositories to Assist AutoML
  Today deep learning is widely used for building software. A software
engineering problem with deep learning is that finding an appropriate
convolutional neural network (CNN) model for the task can be a challenge for
developers. Recent work on AutoML, more precisely neural architecture search
(NAS), embodied by tools like Auto-Keras aims to solve this problem by
essentially viewing it as a search problem where the starting point is a
default CNN model, and mutation of this CNN model allows exploration of the
space of CNN models to find a CNN model that will work best for the problem.
These works have had significant success in producing high-accuracy CNN models.
There are two problems, however. First, NAS can be very costly, often taking
several hours to complete. Second, CNN models produced by NAS can be very
complex that makes it harder to understand them and costlier to train them. We
propose a novel approach for NAS, where instead of starting from a default CNN
model, the initial model is selected from a repository of models extracted from
GitHub. The intuition being that developers solving a similar problem may have
developed a better starting point compared to the default model. We also
analyze common layer patterns of CNN models in the wild to understand changes
that the developers make to improve their models. Our approach uses commonly
occurring changes as mutation operators in NAS. We have extended Auto-Keras to
implement our approach. Our evaluation using 8 top voted problems from Kaggle
for tasks including image classification and image regression shows that given
the same search time, without loss of accuracy, Manas produces models with
42.9% to 99.6% fewer number of parameters than Auto-Keras' models. Benchmarked
on GPU, Manas' models train 30.3% to 641.6% faster than Auto-Keras' models.
