Disk storage management for LHCb based on Data Popularity estimator
  This paper presents an algorithm providing recommendations for optimizing the
LHCb data storage. The LHCb data storage system is a hybrid system. All
datasets are kept as archives on magnetic tapes. The most popular datasets are
kept on disks. The algorithm takes the dataset usage history and metadata
(size, type, configuration etc.) to generate a recommendation report. This
article presents how we use machine learning algorithms to predict future data
popularity. Using these predictions it is possible to estimate which datasets
should be removed from disk. We use regression algorithms and time series
analysis to find the optimal number of replicas for datasets that are kept on
disk. Based on the data popularity and the number of replicas optimization, the
algorithm minimizes a loss function to find the optimal data distribution. The
loss function represents all requirements for data distribution in the data
storage system. We demonstrate how our algorithm helps to save disk space and
to reduce waiting times for jobs using this data.
