A Survey on Green Deep Learning
  In recent years, larger and deeper models are springing up and continuously
pushing state-of-the-art (SOTA) results across various fields like natural
language processing (NLP) and computer vision (CV). However, despite promising
results, it needs to be noted that the computations required by SOTA models
have been increased at an exponential rate. Massive computations not only have
a surprisingly large carbon footprint but also have negative effects on
research inclusiveness and deployment on real-world applications.
  Green deep learning is an increasingly hot research field that appeals to
researchers to pay attention to energy usage and carbon emission during model
training and inference. The target is to yield novel results with lightweight
and efficient technologies. Many technologies can be used to achieve this goal,
like model compression and knowledge distillation. This paper focuses on
presenting a systematic review of the development of Green deep learning
technologies. We classify these approaches into four categories: (1) compact
networks, (2) energy-efficient training strategies, (3) energy-efficient
inference approaches, and (4) efficient data usage. For each category, we
discuss the progress that has been achieved and the unresolved challenges.
