Benchmark Assessment for DeepSpeed Optimization Library
  Deep Learning (DL) models are widely used in machine learning due to their
performance and ability to deal with large datasets while producing high
accuracy and performance metrics. The size of such datasets and the complexity
of DL models cause such models to be complex, consuming large amount of
resources and time to train. Many recent libraries and applications are
introduced to deal with DL complexity and efficiency issues. In this paper, we
evaluated one example, Microsoft DeepSpeed library through classification
tasks. DeepSpeed public sources reported classification performance metrics on
the LeNet architecture. We extended this through evaluating the library on
several modern neural network architectures, including convolutional neural
networks (CNNs) and Vision Transformer (ViT). Results indicated that DeepSpeed,
while can make improvements in some of those cases, it has no or negative
impact on others.
