MIOpen: An Open Source Library For Deep Learning Primitives
  Deep Learning has established itself to be a common occurrence in the
business lexicon. The unprecedented success of deep learning in recent years
can be attributed to: abundance of data, availability of gargantuan compute
capabilities offered by GPUs, and adoption of open-source philosophy by the
researchers and industry. Deep neural networks can be decomposed into a series
of different operators. MIOpen, AMD's open-source deep learning primitives
library for GPUs, provides highly optimized implementations of such operators,
shielding researchers from internal implementation details and hence,
accelerating the time to discovery. This paper introduces MIOpen and provides
details about the internal workings of the library and supported features.
MIOpen innovates on several fronts, such as implementing fusion to optimize for
memory bandwidth and GPU launch overheads, providing an auto-tuning
infrastructure to overcome the large design space of problem configurations,
and implementing different algorithms to optimize convolutions for different
filter and input sizes. MIOpen is one of the first libraries to publicly
support the bfloat16 data-type for convolutions, allowing efficient training at
lower precision without the loss of accuracy.
