Compute and Energy Consumption Trends in Deep Learning Inference
  The progress of some AI paradigms such as deep learning is said to be linked
to an exponential growth in the number of parameters. There are many studies
corroborating these trends, but does this translate into an exponential
increase in energy consumption? In order to answer this question we focus on
inference costs rather than training costs, as the former account for most of
the computing effort, solely because of the multiplicative factors. Also, apart
from algorithmic innovations, we account for more specific and powerful
hardware (leading to higher FLOPS) that is usually accompanied with important
energy efficiency optimisations. We also move the focus from the first
implementation of a breakthrough paper towards the consolidated version of the
techniques one or two year later. Under this distinctive and comprehensive
perspective, we study relevant models in the areas of computer vision and
natural language processing: for a sustained increase in performance we see a
much softer growth in energy consumption than previously anticipated. The only
caveat is, yet again, the multiplicative factor, as future AI increases
penetration and becomes more pervasive.
