PolyScientist: Automatic Loop Transformations Combined with Microkernels
  for Optimization of Deep Learning Primitives
  At the heart of deep learning training and inferencing are computationally
intensive primitives such as convolutions which form the building blocks of
deep neural networks. Researchers have taken two distinct approaches to
creating high performance implementations of deep learning kernels, namely, 1)
library development exemplified by Intel MKL-DNN for CPUs, 2) automatic
compilation represented by the TensorFlow XLA compiler. The two approaches have
their drawbacks: even though a custom built library can deliver very good
performance, the cost and time of development of the library can be high.
Automatic compilation of kernels is attractive but in practice, till date,
automatically generated implementations lag expert coded kernels in performance
by orders of magnitude.
  In this paper, we develop a hybrid solution to the development of deep
learning kernels that achieves the best of both worlds: the expert coded
microkernels are utilized for the innermost loops of kernels and we use the
advanced polyhedral technology to automatically tune the outer loops for
performance. We design a novel polyhedral model based data reuse algorithm to
optimize the outer loops of the kernel. Through experimental evaluation on an
important class of deep learning primitives namely convolutions, we demonstrate
that the approach we develop attains the same levels of performance as Intel
MKL-DNN, a hand coded deep learning library.
