WeNet: Weighted Networks for Recurrent Network Architecture Search
  In recent years, there has been increasing demand for automatic architecture
search in deep learning. Numerous approaches have been proposed and led to
state-of-the-art results in various applications, including image
classification and language modeling. In this paper, we propose a novel way of
architecture search by means of weighted networks (WeNet), which consist of a
number of networks, with each assigned a weight. These weights are updated with
back-propagation to reflect the importance of different networks. Such weighted
networks bear similarity to mixture of experts. We conduct experiments on Penn
Treebank and WikiText-2. We show that the proposed WeNet can find recurrent
architectures which result in state-of-the-art performance.
