NASIB: Neural Architecture Search withIn Budget
  Neural Architecture Search (NAS) represents a class of methods to generate
the optimal neural network architecture and typically iterate over candidate
architectures till convergence over some particular metric like validation
loss. They are constrained by the available computation resources, especially
in enterprise environments. In this paper, we propose a new approach for NAS,
called NASIB, which adapts and attunes to the computation resources (budget)
available by varying the exploration vs. exploitation trade-off. We reduce the
expert bias by searching over an augmented search space induced by
Superkernels. The proposed method can provide the architecture search useful
for different computation resources and different domains beyond image
classification of natural images where we lack bespoke architecture motifs and
domain expertise. We show, on CIFAR10, that itis possible to search over a
space that comprises of 12x more candidate operations than the traditional
prior art in just 1.5 GPU days, while reaching close to state of the art
accuracy. While our method searches over an exponentially larger search space,
it could lead to novel architectures that require lesser domain expertise,
compared to the majority of the existing methods.
