Graph-based Neural Architecture Search with Operation Embeddings
  Neural Architecture Search (NAS) has recently gained increased attention, as
a class of approaches that automatically searches in an input space of network
architectures. A crucial part of the NAS pipeline is the encoding of the
architecture that consists of the applied computational blocks, namely the
operations and the links between them. Most of the existing approaches either
fail to capture the structural properties of the architectures or use
hand-engineered vector to encode the operator information. In this paper, we
propose the replacement of fixed operator encoding with learnable
representations in the optimization process. This approach, which effectively
captures the relations of different operations, leads to smoother and more
accurate representations of the architectures and consequently to improved
performance of the end task. Our extensive evaluation in ENAS benchmark
demonstrates the effectiveness of the proposed operation embeddings to the
generation of highly accurate models, achieving state-of-the-art performance.
Finally, our method produces top-performing architectures that share similar
operation and graph patterns, highlighting a strong correlation between the
structural properties of the architecture and its performance.
