Dissecting FLOPs along input dimensions for GreenAI cost estimations
  The term GreenAI refers to a novel approach to Deep Learning, that is more
aware of the ecological impact and the computational efficiency of its methods.
The promoters of GreenAI suggested the use of Floating Point Operations (FLOPs)
as a measure of the computational cost of Neural Networks; however, that
measure does not correlate well with the energy consumption of hardware
equipped with massively parallel processing units like GPUs or TPUs. In this
article, we propose a simple refinement of the formula used to compute floating
point operations for convolutional layers, called {\alpha}-FLOPs, explaining
and correcting the traditional discrepancy with respect to different layers,
and closer to reality. The notion of {\alpha}-FLOPs relies on the crucial
insight that, in case of inputs with multiple dimensions, there is no reason to
believe that the speedup offered by parallelism will be uniform along all
different axes.
