Path-Level Network Transformation for Efficient Architecture Search
  We introduce a new function-preserving transformation for efficient neural
architecture search. This network transformation allows reusing previously
trained networks and existing successful architectures that improves sample
efficiency. We aim to address the limitation of current network transformation
operations that can only perform layer-level architecture modifications, such
as adding (pruning) filters or inserting (removing) a layer, which fails to
change the topology of connection paths. Our proposed path-level transformation
operations enable the meta-controller to modify the path topology of the given
network while keeping the merits of reusing weights, and thus allow efficiently
designing effective structures with complex path topologies like Inception
models. We further propose a bidirectional tree-structured reinforcement
learning meta-controller to explore a simple yet highly expressive
tree-structured architecture space that can be viewed as a generalization of
multi-branch architectures. We experimented on the image classification
datasets with limited computational resources (about 200 GPU-hours), where we
observed improved parameter efficiency and better test results (97.70% test
accuracy on CIFAR-10 with 14.3M parameters and 74.6% top-1 accuracy on ImageNet
in the mobile setting), demonstrating the effectiveness and transferability of
our designed architectures.
