Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A
  Data-Centric Approach Using MAESTRO
  The data partitioning and scheduling strategies used by DNN accelerators to
leverage reuse and perform staging are known as dataflow, and they directly
impact the performance and energy efficiency of DNN accelerator designs. An
accelerator microarchitecture dictates the dataflow(s) that can be employed to
execute a layer or network. Selecting an optimal dataflow for a layer shape can
have a large impact on utilization and energy efficiency, but there is a lack
of understanding on the choices and consequences of dataflows, and of tools and
methodologies to help architects explore the co-optimization design space. In
this work, we first introduce a set of data-centric directives to concisely
specify the space of DNN dataflows in a compilerfriendly form. We then show how
these directives can be analyzed to infer various forms of reuse and to exploit
them using hardware capabilities. We codify this analysis into an analytical
cost model, MAESTRO (Modeling Accelerator Efficiency via Spatio-Temporal Reuse
and Occupancy), that estimates various cost-benefit tradeoffs of a dataflow
including execution time and energy efficiency for a DNN model and hardware
configuration. We demonstrate the use of MAESTRO to drive a hardware design
space exploration (DSE) experiment, which searches across 480M designs to
identify 2.5M valid designs at an average rate of 0.17M designs per second,
including Pareto-optimal throughput- and energy-optimized design points.
