Exploring the Design Space of Deep Convolutional Neural Networks at
  Large Scale
  In recent years, the research community has discovered that deep neural
networks (DNNs) and convolutional neural networks (CNNs) can yield higher
accuracy than all previous solutions to a broad array of machine learning
problems. To our knowledge, there is no single CNN/DNN architecture that solves
all problems optimally. Instead, the "right" CNN/DNN architecture varies
depending on the application at hand. CNN/DNNs comprise an enormous design
space. Quantitatively, we find that a small region of the CNN design space
contains 30 billion different CNN architectures.
  In this dissertation, we develop a methodology that enables systematic
exploration of the design space of CNNs. Our methodology is comprised of the
following four themes.
  1. Judiciously choosing benchmarks and metrics.
  2. Rapidly training CNN models.
  3. Defining and describing the CNN design space.
  4. Exploring the design space of CNN architectures.
  Taken together, these four themes comprise an effective methodology for
discovering the "right" CNN architectures to meet the needs of practical
applications.
