KD-Lib: A PyTorch library for Knowledge Distillation, Pruning and
  Quantization
  In recent years, the growing size of neural networks has led to a vast amount
of research concerning compression techniques to mitigate the drawbacks of such
large sizes. Most of these research works can be categorized into three broad
families : Knowledge Distillation, Pruning, and Quantization. While there has
been steady research in this domain, adoption and commercial usage of the
proposed techniques has not quite progressed at the rate. We present KD-Lib, an
open-source PyTorch based library, which contains state-of-the-art modular
implementations of algorithms from the three families on top of multiple
abstraction layers. KD-Lib is model and algorithm-agnostic, with extended
support for hyperparameter tuning using Optuna and Tensorboard for logging and
monitoring. The library can be found at - https://github.com/SforAiDl/KD_Lib.
