Fast Task-Aware Architecture Inference
  Neural architecture search has been shown to hold great promise towards the
automation of deep learning. However in spite of its potential, neural
architecture search remains quite costly. To this point, we propose a novel
gradient-based framework for efficient architecture search by sharing
information across several tasks. We start by training many model architectures
on several related (training) tasks. When a new unseen task is presented, the
framework performs architecture inference in order to quickly identify a good
candidate architecture, before any model is trained on the new task. At the
core of our framework lies a deep value network that can predict the
performance of input architectures on a task by utilizing task meta-features
and the previous model training experiments performed on related tasks. We
adopt a continuous parametrization of the model architecture which allows for
efficient gradient-based optimization. Given a new task, an effective
architecture is quickly identified by maximizing the estimated performance with
respect to the model architecture parameters with simple gradient ascent. It is
key to point out that our goal is to achieve reasonable performance at the
lowest cost. We provide experimental results showing the effectiveness of the
framework despite its high computational efficiency.
