{'fraction_non_alphanumeric': 0.03498542274052478, 'fraction_numerical': 0.01817505117548539, 'mean_word_length': 5.167559296097934, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Predictive models for COVID-19 detection using routine blood tests and machine learning

The problem of accurate, fast, and inexpensive COVID-19 tests has been urgent till now. Standard COVID-19 tests need high-cost reagents and specialized laboratories with high safety requirements, are time-consuming. Data of routine blood tests as a base of SARS-CoV-2 invasion detection allows using the most practical medicine facilities. But blood tests give general information about a patient’s state, which is not directly associated with COVID-19. COVID-19-specific features should be selected from the list of standard blood characteristics, and decision-making software based on appropriate clinical data should be created. This review describes the abilities to develop predictive models for COVID-19 detection using routine blood tests and machine learning.


Introduction
The problem of accurate, fast, and inexpensive COVID-19 tests has been urgent till now. The key message of this review is that blood tests contain enough information to detect SARS-CoV-2 infected patients with acceptable accuracy. To establish the corresponding latent dependencies between blood test parameters and COVID-19 presence, a machine learning approach should be applied. The most effective prediction models with accuracy of COVID-19 diagnosis of about 85-95% are based on using Random forest, Gradient boosting, and their variations. Surprisingly, that these models use not more than only 8-10 blood parameters. A set of the most informative features includes mean corpuscular hemoglobin, lymphocytes, leukocytes, basophils, eosinophils, C-reactive protein, bilirubin, D-dimer.
Current "gold standard" for SARS-CoV-2 infection identification is based on a transcription-polymerase chain reaction (RT-PCR) and Enzyme-linked immunosorbent assay (ELISA) tests [1,2]. RT-PCR has high specificity but low sensitivity [3,4] and high misclassification in the early stage of disease [5]. The drawbacks are high-cost equipment and consumables, long time of analysis. Additional information can be achieved by computer tomography or X-ray visualization [6,7,8], but these methods require high-tech equipment and cannot be used often. A possible solution is to use more conventional clinical tests combined with sophisticated but powerful analytical methods from the field of Artificial Intelligence implemented as a Medical Decision Support System (MDS). The attractiveness of this idea is that Artificial Intelligence can compensate a nonspecificity of general clinical tests.
Since the COVID-19 outbreak at the end of 2019, several Medical Decision Support Systems (MDSSs) were developed for this disease diagnostic and severity prediction [9,10,11,12,13]. MDSSs can be based on data-driven, knowledge-driven, and hybrid (a combination of knowledge driven and data-based) data models. Data driven models (DDM) rely only on the observed data, while knowledge-driven (KDM) ones use human logic. The hybrid models take advantage of both approaches. For example, DDMs are superior in natural language processing and can extract knowledge from Electronic Health Records, and KDM can be used for the support of disease diagnostics. Usually, DDM MDSSs use machine learning (ML) algorithms like artificial neural networks; KDM MDSSs are based on an expert knowledge and human logic [14]. Most trustworthy are expert systems based on subjective evaluations described as medical treatment protocols. Such systems are the state-of-the-art for a well-known disease [15] but lack the knowledge with COVID-19. ML MDSSs are available to analyze latent relations in data and construct complex predictive models, but this is a purely data-driven approach and couldn't handle data uncertainty.
ML pipeline is a sequence of algorithms that includes preprocessing, feature extraction/selection, prediction model construction, and its validation (see Figure 1) [13]. A brief description of these steps is presented below.

Preprocessing
The necessity of preprocessing depends on the quality of input data. Routine blood test results often have missing values, and imputation algorithms based on the statistical mean are used to fill it. Also, detection and removal of outliers, presenting data, significantly different from the major of the dataset, should be conducted. The statistical-based approaches like a parametric Z-score [4,16], isolation forest [17], density-based spatial clustering of applications with noise [18] are considered to be the most effective.

Feature extraction
The feature extraction is based on a transformation of initial parameters to a feature vector of less dimension due to removing redundant features. Principal component analysis and multidimensional scaling are the most frequently used [19,20].

Feature selection
Feature selection is based on uninformative parameters removing. There are univariate and multivariate methods of feature selection [21,22]. Univariate methods analyze every feature impact on the predictive model accuracy independently. Multivariate methods do the same for several features jointly. Some classification methods, like Decision trees (DT) and Random forests (RF) can also estimate features' informativeness [23].

Prediction model construction
This step allows making a conclusion about a studied object belonging to one of the preliminarily determined classes. Supervised ML methods useful for creating prediction models in medicine are limited by the requirements of model transparency and a small volume of medical datasets [24]. DT, RF, Support vector machine (SVM), and gradient boosting are the most often used [24,25].

Prediction model validation
The validation needs to estimate a prediction model accuracy. It should be conducted with a data set, which was not used for data model training. The latter most simple implementation is train-test split [19,26]. Another option is to get a small standalone dataset from the other source and use it as a final performance measure [27]. K-fold cross-validation is often used [19].
Using routine blood tests as an additional diagnostic tool is very attractive because of its simplicity and availability in all medical institutions of practical healthcare. The routine blood tests issue is that they give general information about a patient's state, which is not directly associated with COVID-19 presence or absence. COVID-19-specific features should be selected from the list of standard blood characteristics, and decision-making software based on appropriate clinical data should be created to overcome this issue. The latter can be implemented using ML.
This review aims to analyze routine blood test availability for COVID-19 detection and the ways of ML pipeline implementation using routine blood test data.

The blood test informative features selection
We analyzed about 50 articles published in journals in the field of medicine and machine learning and indexed in Scopus, Google Scholar, and Web of Science. The search included the following terms: COVID-19; Blood tests; Machine learning, Statistical analysis.
The features met in more than 10 papers and features considered by the authors as informative are presented in Table 1.
The oxygen desaturation is associated with severe respiratory failure in COVID-19 patients. SARS-CoV-2 surface glycoproteins bind to porphyrin on hemoglobin and inhibit heme metabolism. It is a possible mechanism of MCH and MCHC changing during COVID-19 [59].
Lymphopenia (temporary or persistent decrease in the level of lymphocytes in the blood) is a typical symptom for COVID-19 patients [60]. An essential decreasing CD8 þ T cells, a type of lymphocytes, was established for SARS-CoV-2 infected patients compared to vaccinated subjects [61].
Eosinophils play an essential role in SARS-CoV-2 infection associated with high CD62L expression, a lung eosinophil marker [62]. The basophil    CRP C-reactive protein C-reactive protein is produced by the liver and is induced by various inflammatory mediators, such as interleukin-6. In a healthy person, it is absent or present in minimal amounts. Therefore, it is an early marker of an acute phase of inflammatory response. level is correlated with humoral response to SARS-CoV-2 infection [62,63]. CRP is usually produced by hepatocytes and, additionally, by macrophages in a place of inflammation [64]. The latter is strongly related to COVID-19 disease complications and mortality. A cytokine storm appearing at the most dangerous stage of COVID-19 pneumonia stimulates hepatocytes to produce CRP [65,66]. CRP is considered an independent indicator of COVID-19 severity [64,67,68]. Aspartate transaminase is a statistically significant marker of pulmonary fibrosis caused by SarS-CoV-2 infection [69]. SARS-CoV-2-positive patients (positive test for SARS-CoV-2) showed a statistically significant decrease in calcium concentration compared to SARS-CoV-2-negative patients [70]. High levels of ferritin, as well as severe lymphopenia (decrease in T-, Blymphocytes), also high levels of C-reactive protein, ferritin, D-dimer, ALT and AST, are signs of a cytokine storm [71]. High ferritin levels were found in autopsies of deceased COVID-19 patients [72].
A high bilirubin content was established in severe Covid-19 patients compared to milder forms (an averaged difference was in interval from 0.27 to 0.95 μmol/L) [73,74,75].
As mentioned above, D-dimer is a biomarker of thrombosis. A 3-4 fold D-dimer concentration increasing was associated with poor prognosis [76].

The COVID-19 prediction models construction using blood test data
The dataset description used by the authors, number of selected/ extracted informative features, and created COVID-19 prediction model performance are shown in Table 2.
Accuracy, sensitivity (SE), and specificity (SP) are defined as follows. Accuracy reflects how much the prediction is exact [36]: The sensitivity is an ability of a method identifying true positives, and specificity (SP) is an ability to identify true negatives [44,49]: Here: -     Remarks and abbreviations. created data model and excellent accuracy. In 9 cases, the DT-based methods demonstrated the best prediction accuracy. A RF-based demonstrated COVID-19 diagnosis accuracy of 92.9% [76]. The SVM generalization potential and good performance are the reasons of its frequent use [26,31,37,43,51]. The Boosting family is in the third place in popularity and in the second place of prediction accuracy (7 studies). The Logistic regression is effective for informative features' extraction and robust to overfitting. Researchers are still using KNN and Naïve Bayes (NB), which were among the early appeared supervised learning methods and do not show high results. An exception is the Bayesian network with diagnosis performance of 95.2% [76].
Researchers are still aware of ANN and Deep Neural Network (DNN) usage because the number of features in raw data is small (tens) compared to a number of samples (hundreds), which is not suitable for these methods.
A predictive model accuracy depends on the data volume and quality. In the analyzed papers, the smallest dataset included 51 COVID-19 patients, the largest -7335 ones [32]. The average number of COVID-19 patients was 919. The most large control group included 125,542 samples [50]. The average number of informative blood features was 8; the median was 6. The mean accuracy was 88%. The simplicity of the best predictive models is an argument that overfitting was avoided. A relation between the number of informative features and a predictive model accuracy is shown in Figure 3.

Discussion
Circulating blood contains the metabolic biomarkers of inflammatory processes in a human body, so the metabolic profile potentially can be used for COVID-19 diagnostics and severe prediction. It can be seen from   Table 1 that the most full profile contains up to a hundred metabolites. Removing redundant ones can significantly improve the robustness of the predictive models on the one hand and reveal the most informative blood features for diagnostic purposes. Preprocessing of input data includes missing data imputations based on statistics. Feature extraction was not usually used. The used machine learning systems provide the following result: there is a definite relation between mean corpuscular hemoglobin, lymphocytes, leukocytes, basophils, eosinophils, C-reactive protein, bilirubin, D-dimer, and COVID-19 (see Table 1). The most effective prediction models were based on using Random forest, Gradient boosting, and their variations. The predictive model achieved the highest accuracy score with a relatively small dataset size can be a subject of overfitting. Datasets with over 500 samples demonstrate an accuracy of about 85-95%, that can be considered as very good (see Table 2). Such results should be deemed positive, and specific blood metabolites-oriented diagnostics may be used in the clinical practice.
Significant progress in blood data analysis became available due to open datasets, also cited in this paper (see Table 2). Though merging the small datasets into large ones can benefit data scientists, it is still a problem from the ethical and medical points of view. Federative machine learning (for instance, OpenFL project by Intel Corporation ©) can solve a problem with data security, but the medical protocol unification issue is persisting. Different machine learning pipelines and the absence of the standard validation strategy that proves the quality of the predictive data model diminish the trust in such results. The open-source software, which allows the reproduction of achieved results, is a key to highquality data models.

Conclusion
PCR test is still the best choice for diagnosing SARS-COV-2 infection [80] but, despite its usefulness, it requires expensive laboratory equipment, highly qualified laboratory personnel, and the time between sampling and the result is relatively long. Other clinical and immunodiagnostic tests should also be taken into account when interpreting the results of RT-PCR tests for a more reliable diagnosis [81]. It is necessary to improve RT-PCR tests to decrease false-negative and false-positive results [82]. COVID-19 pandemic force rapid development of a new method to study disease metabolic pathways.
Routine blood tests, including microfluidic devices, are minimally invasive, minimizing the sample volume to one drop. From this perspective, proposed predictive models allow us to understand the connections among human blood metabolites, COVID-19 diagnosis and prognosis of severe. Such knowledge is not a clinic-ready system, but a step before it. Once the verified set of metabolites is found, a simple expert knowledge-based system can be constructed to be used routinely.
Some ideas for the future development of COVID-19 blood tests are as follows. Classes imbalance can lead to a biased predictive model. Synthetic Minority Over-sampling Technique allows generating additional data samples through variations of each blood feature [28,31,33]. A game theory-based Shapley value method can provide a reliable, feature extraction [83,84] An ensemble learning based on combining several classification algorithms and generating a final prediction, for instance, by the voting procedure allows improving a predictive model performance [38,54,85,86,87,88].
Another achievement of such studies is the proof that machine learning methods coupled with thoroughly developed protocols and large datasets can provide rapid and reliable techniques to generate fundamental knowledge about socially significant diseases. We encourage scientific groups to share their datasets and develop machine learning models because humanity can overcome the pandemic by uniting efforts.
The essential proofs, which may lead to a clinical trial and further introduction of such methods into the clinical practice, are improvements and conclusions by physiologists found in many reviewed papers.

Author contribution statement
All authors listed have significantly contributed to the development and the writing of this article.