{'fraction_non_alphanumeric': 0.04330161816792833, 'fraction_numerical': 0.01454962150149316, 'mean_word_length': 4.7864175205947355, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
COVID-DA: Deep Domain Adaptation from Typical Pneumonia to COVID-19

The outbreak of novel coronavirus disease 2019 (COVID-19) has already infected millions of people and is still rapidly spreading all over the globe. Most COVID-19 patients suffer from lung infection, so one important diagnostic method is to screen chest radiography images, e.g., X-Ray or CT images. However, such examinations are time-consuming and labor-intensive, leading to limited diagnostic efficiency. To solve this issue, AI-based technologies, such as deep learning, have been used recently as effective computer-aided means to improve diagnostic efficiency. However, one practical and critical difficulty is the limited availability of annotated COVID-19 data, due to the prohibitive annotation costs and urgent work of doctors to fight against the pandemic. This makes the learning of deep diagnosis models very challenging. To address this, motivated by that typical pneumonia has similar characteristics with COVID-19 and many pneumonia datasets are publicly available, we propose to conduct domain knowledge adaptation from typical pneumonia to COVID-19. There are two main challenges: 1) the discrepancy of data distributions between domains; 2) the task difference between the diagnosis of typical pneumonia and COVID-19. To address them, we propose a new deep domain adaptation method for COVID-19 diagnosis, namely COVID-DA. Specifically, we alleviate the domain discrepancy via feature adversarial adaptation and handle the task difference issue via a novel classifier separation scheme. In this way, COVID-DA is able to diagnose COVID-19 effectively with only a small number of COVID-19 annotations. Extensive experiments verify the effectiveness of COVID-DA and its great potential for real-world applications.

COVID-19 (including 202,597 deaths, with a fatal rate of 6.9%) [3], leading to great threats for global public health. Due to COVID-19, many countries have been forced into emergencies [4] and suffered from devastating effects on population health [5] and social economy [6], [7]. To fight against COVID-19, one key step is to diagnose patients and provide immediate medical treatment, thereby preventing further spread of COVID-19.
Currently, the main diagnosis method for COVID-19 is the real-time Reverse-Transcriptase Polymerase Chain Reaction (RT-PCR) [8] test, which is regarded as the gold standard for COVID-19 detection. However, RT-PCR has a lower diagnostic sensitivity and generally requires repeated tests for the final confirmation of infection [9]. As a result, the RT-PCR test is very time-consuming and laborious [10]. Meanwhile, it is extremely difficult for hospitals in hyper-endemic regions to provide sufficient RT-PCR tests for tons of suspected patients. To conquer this issue, an alternative diagnostic method is based on the screening of chest radiography images (CRIs), e.g., X-ray or computed tomography (CT) images [11], since COVID-19 patients often present abnormal characteristics of lung infection on CRIs [12], [13]. Compared with RT-PCR, CRI-based diagnosis method is more efficient and has been widely used in clinical diagnosis in practice [14], [27]. Nevertheless, when dealing with tons of patients, medical specialists still need to screen CRIs one by one, which, however, is highly stressful and time-consuming. Hence, there is an urgent need to develop computer-aided methods for the diagnosis of COVID-19, which help to improve the diagnostic efficiency of medical specialists.
Recently, deep learning (DL) has achieved remarkable success in medical image analysis [15]- [21]. It is a natural idea to develop DL-based methods for COVID-19 diagnosis. One of the key factors behind the success of DL is the large amount of labeled data [22]. However, in the diagnosis of COVID-19, such extensive annotations are unavailable now due to prohibitive annotation costs and urgent work of doctors to fight against the pandemic. Hence, there is a strong motivation to develop domain adaptation [23] to improve DL-based diagnostic models for COVID-19. Specifically, domain adaptation leverages a source domain with rich labeled data to help the model learning on the target domain. Considering that typical pneumonia has some similar characteristics with COVID- 19 and many open-source pneumonia datasets are accessible, we seek to conduct domain adaptation from typical pneumonia to COVID-19 in this paper.

arXiv:2005.01577v1 [eess.IV] 30 Apr 2020
In this task, there are two major challenges. The first one is the domain discrepancy of data distributions, which mainly derived from different medical imaging devices or techniques. In this regard, directly applying a deep model trained on the typical pneumonia domain to the COVID-19 domain tends to perform poorly and be impractical. The second challenge lies in the diagnostic task difference between typical pneumonia and COVID-19. The two tasks are similar but not completely the same, which may result in poor generalization of the source-trained classifier to the target domain. However, most existing domain adaptation methods [24]- [26] neglect the task difference issue and adopt only one domain-shared classifier. As a result, they may perform poorly in COVID-19 diagnosis.
To solve the above challenges, we propose a novel deep domain adaptation method for the diagnosis of COVID-19 (namely COVID-DA), which relies on domain adversarial learning and a new classifier separation scheme. To be specific, we alleviate the domain discrepancy by aligning the feature distributions of two domains via feature adversarial adaptation. In this way, COVID-DA is able to learn domaininvariant features for classification. Based on such features, we handle the task difference issue based on a novel classifier separation scheme, which disentangles the classifier into a domain-shared classifier and two domain-specific classifiers (for two domains). Specifically, the domain-shared classifier aims to learn task-shared classification knowledge between typical pneumonia and COVID-19, while the domain-specific classifiers seek to learn task-specific classification knowledge. To this end, we train the domain-shared classifier to learn taskshared semantic information by aligning the joint distributions of two domains over features and predictions. Meanwhile, we maximize the diversity between the domain-shared and domain-specific classifiers to make the latter ones focus on task-specific classification information. Based on the above, COVID-DA is able to diagnose COVID-19 effectively with only a limited amount of labeled data, and thus is more applicable in real-world applications.
Our main contributions are summarized as follows: • We propose a novel deep domain adaptation method for the diagnosis of COVID-19. To the best of our knowledge, this is the first attempt to study domain adaptation from typical pneumonia to COVID-19. • Based on a novel classifier separation scheme and a new domain adversarial adaptation method, the proposed method is able to overcome the task difference and domain discrepancy simultaneously. • We conduct extensive experiments to evaluate the proposed method. Promising results demonstrate its effectiveness and superiority, e.g., the proposed method improves the diagnostic performacne for COVID-19 from 0.6875 to 0.9298 in terms of the F1 metric.
The rest of this paper is organized as follows. We first present related work in Section II. Following that, we detail the problem definition and the proposed method in Section III. Next, we empirically evaluate the proposed method in Section IV, and conclude the paper in Section V.

A. Computer-aided Diagnosis for COVID-19
To control the transmission of COVID-19, one of the most important steps is to screen out the infected patients, and then provide proper treatments for them. Due to the relatively low time and labour costs, chest radiography imaging (CRI), e.g., X-ray or computed tomography (CT) imaging, has been widely adopted to provide diagnostic evidences for radiologists. However, when facing tons of suspected patients, it is still time-consuming for radiologists to screen medical images one by one, leading to inferior diagnostic efficiency. To address this, based on deep learning techniques, many computer-aided diagnosis methods have been developed [14], and some of them have been deployed in hospitals [27]. For example, Xu et al. proposed a deep learning (DL) method for the early detection of COVID-19 from Influenza-A viral pneumonia and normal cases [28]. Chen et al. proposed a UNet++ based deep model for segmenting the infected regions of COVID-19 [29]. Nevertheless, since deep models are notoriously data-hungry, these DL-based methods require plenty of annotated data to achieve satisfactory performance. However, in the diagnosis of COVID-19, such rich supervision is unavailable in most practical scenarios due to prohibitive annotation costs and urgent work of doctors to fight the pandemic.
To solve this issue, recent studies [10], [11] directly combined publicly available typical pneumonia datasets and COVID-19 dataset together to train a multi-class classification model. However, these methods ignore the domain discrepancy between typical pneumonia and COVID-19, thereby resulting in limited diagnostic performance for COVID-19. Therefore, there is an urgent need to develop task-specific domain adaptation methods for COVID-19 to improve the performance of DL-based diagnosis models.

B. Domain Adaptation
Most existing domain adaptation methods for natural images seek to alleviate the domain discrepancy either by adding adaptation layers to match high-order moments of distributions, e.g., DDC [26], or by devising a domain discriminator to learn domain-invariant features in an adversarial manner, e.g., DANN [30] and MCD [31]. Following the latter manner, CLAN [32] proposed to conduct category-aware domain adaptation instead of only global alignment of domain distributions. In medical image analysis, by taking the characteristics of medical imaging into account, Kamnitsas et al. attempted to introduce a multi-connected domain discriminator for improved adversarial training [33]. Ren et al. proposed a Siamese architecture on the target domain to add a regularization for the whole-slide images [34]. However, all these methods ignore the task difference between domains, and thus may perform poorly on the adaptation from typical pneumonia to COVID-19. To handle these, we propose a new deep domain adaptation method for COVID-19, which aims to alleviate the domain discrepancy and overcome task difference simultaneously. In this way, the proposed method is able to diagnose COVID-19 more effectively in real-world applications.
be limited labeled target data and D t u ={x k } n t u k=1 be unlabeled target data. Here, n s , n t l , n t u denote the number of source data, labeled target data and unlabeled target data, where n t l n t u and n t l n s . Moreover, let D t =D t l ∪D t u be the complete target domain with n t =n t l +n t u as the sample number. The goal is to learn a well-performed deep model for the target domain, using both source samples (labeled) and target samples (partially labeled). This task, however, is very difficult due to (1) only limited labeled samples in the target domain and (2) apparent discrepancy between typical Pneumonia and COVID-19 in terms of domain distributions and tasks. However, existing domain adaptation methods for medical images only focus on alleviating the discrepancy in terms of domain distributions, while ignoring the task difference between domains. As a result, directly applying them to the task tends to perform poorly in practice. To solve this, we propose a new deep domain adaptation method for the diagnosis of COVID-19, namely COVID-DA.

A. Overall Scheme of COVID-DA
To enforce effective domain knowledge adaptation, we seek to alleviate the domain discrepancy with domain adversarial adaptation and handle the task difference via a novel classifier separation scheme. To this end, as shown in Fig. 1, COVID-DA consists of three main parts: (1) a domain-shared feature extractor G f for extracting domain-invariant features; (2) two domain discriminators {D 1 , D 2 } for feature adaptation and classifier adaptation, respectively; (3) a domain-shared classifier C d and two domain-specific classifiers {C t , C s } for the diagnosis of COVID-19. Note that the separation of domain-shared and domain-specific classifiers helps to disentangle task-shared and task-specific pathological information regarding typical pneumonia and COVID-19.
Overall, COVID-DA conducts three main strategies as follows. (a) feature adversarial adaptation: we impose a domain loss L d1 to align the feature distributions of two domains, so that the domain discrepancy is minimized in an adversarial learning manner [24], [25]; (b) classifier adversarial adaptation: we exploit a domain loss L d2 to conduct joint distribution alignment for the domain-shared classifier C d , making it able to learn domain-shared pathological information in an adversarial learning manner; (c) classifier diversity maximization: we maximize the diversity between the domain-shared and domain-specific classifiers via a diversity loss L div , so that the domain-specific classifiers can learn task-specific information in two domains. Note that the strategies (b) and (c) help COVID-DA to handle the task difference effectively. Lastly, we train the feature extractor and all classifiers via a focal classification loss L f [35], which makes the model class imbalance-aware and discriminative. In this way, COVID-DA is able to adapt the source domain knowledge to the target domain and diagnose COVID-19 effectively.
The overall training procedure of COVID-DA is to solve the following minimax problem [36]: where Θ 1 ={θ f , θ c d , θ ct , θ cs } denotes the parameters of the feature extractor G f and all classifiers {C d , C t , C s }, while Θ 2 ={θ d1 , θ d2 } denotes the parameters of two discriminators {D 1 , D 2 }. Here, α and β are trade-off parameters.
We next detail two domain losses L d1 and L d2 in Section III-B and Section III-C, respectively. Following that, we detail the classifier diversity loss L div in Section III-D and then the focal loss L f in Section III-E.

B. Feature Adversarial Adaptation
Diverse imaging devices and preprocessing techniques intrinsically result in huge domain discrepancy in terms of data distribution. To resolve the discrepancy, we resort to domain adversarial learning for aligning feature distributions of domains. Specifically, on the one hand, a domain discriminator D 1 is trained to adequately distinguish feature representations between two domains by minimizing a domain loss L d1 . On the other hand, the feature extractor G f is trained to confuse the discriminator by maximizing the domain loss L d1 . As a result, the learned feature extractor is able to extract domaininvariant features that confuse the discriminator well. Based on the least square distance [37], we define the domain loss for feature adversarial adaptation as: where d 1 (x) = D 1 (G f (x)) denotes the prediction of the domain discriminator w.r.t. x. We set the label of the target domain to 1 and that of the source domain to 0.

C. Classifier Adversarial Adaptation
Most existing adversarial domain adaptation methods [24], [30], [38] assume that the source domain deals with the same classification task as the target domain. Hence, they usually focus on feature distribution alignment as Section III-B and use a classifier trained on the source domain to classify the target data. However, they may fail to handle the problem in this paper, since pneumonia diagnosis and COVID-19 diagnosis are similar but not completely the same, originated from different pathological mechanisms. Specifically, let P(f s ) and P(f t ) denote the feature distributions of the source and target domains, while let P(y s |f s ) and P(y t |f t ) be the prediction conditional distributions of two domains. Even though the feature distributions have been matched (i.e., P(f s )=P(f t )), the task difference potentially results in different prediction conditional distributions (i.e., P(y s |f s ) =P(y t |f t )) [39], [40]. As a result, only using the source-trained classifier P(y s |f s ) may not be able to diagnose COVID-19 well.
To solve this issue, as shown in Fig. 1, we propose a novel classifier separation scheme by disentangling the domain-shared classifier C d and the domain-specific classifiers {C s , C t }. To be specific, the domain-shared classifier seeks to acquire task-shared classification knowledge, while the domain-specific classifiers aim to learn task-specific knowledge. By taking the average ensemble of two classifiers as the final prediction [41], COVID-DA is able to handle the task difference and diagnose COVID-19 well.
In this scheme, one key issue is how to learn the domainshared classifier. In fact, when the feature distributions match well (i.e., P(f s )=P(f t )), if we train the classifier to align the joint distributions (i.e., P(y s , f s )=P(y t , f t )), then the domainshared classifier is able to learn task-shared classification knowledge since P(y, f )=P(f )P(y|f ) [42]. Motivated by this, we propose to train the domain-shared classifier by aligning the joint distributions via domain adversarial learning [25].
On the one hand, a discriminator D 2 is trained to adequately differentiate the joint distributions between domains by minimizing a domain loss L d2 . Specifically, the input of the discriminator D 2 consists of both features and predictions. On the other hand, the domain-shared classifier C d is trained to confuse the discriminator by maximizing the domain loss. As Section III-B, we define the domain loss for classifier adversarial adaptation based on the least square distance: ) denotes the prediction of the domain discriminator regarding the joint distribution over the feature and the prediction w.r.t. x. Moreover, we denote the label of the target domain as 1 and that of the source as 0.

D. Classifier Diversity Maximization
In COVID-DA, we handle the issue of task difference by disentangling the domain-shared and domain-specific classifiers. In Section III-C, we have enforced the domain-shared classifier to acquire domain-invariant classification knowledge. In this section, we further enforce the domain-specific classifiers to acquire domain private classification knowledge. To this end, we maximize the diversity between the domainspecific classifier and the domain-shared classifier. Specifically, given any sample x, letŷ d denote the prediction of domain-shared classifiers (i.e., C d ), and letŷ t andŷ s denote the prediction of the domain-specific classifier (i.e., C t and C s ). Based on the cosine distance, we maximize the classifier diversity based on the following diversity loss: In this way, the two domain-specific classifiers are different with the domain-shared classifier as large as possible, and thus able to learn domain-specific classification information. Moreover, since the final prediction of COVID-DA is the average ensemble of two classifiers, maximizing classifier diversity also enhance the performance of ensemble learning [43].

E. Focal Loss for COVID-19 Diagnosis
For the diagnosis of COVID-19, one can adopt any classification losses to train our COVID-DA, e.g., cross-entropy. Nevertheless, considering the class imbalance issue in medical diagnosis, we use the focal loss [35] as follows: whereŷ is the final prediction of COVID-DA w.r.t. a given sample x. Here,ŷ is the average prediction of both domainshared (C d ) and domain-specific (C t or C s ) classifiers. Note that, the focal loss is a widely-used loss for class imbalance issue [24]. Moreover, denotes the element-wise product and γ is a hyper-parameter in focal loss.
We summarize the training and inference details of COVID-DA in Algorithms 1 and 2, respectively. Moreover, we implement the adversarial learning via a gradient reversal layer (GRL) [30], [38], which reverses the gradient of the domain loss when backpropagating to the feature extractor or domainshared classifier. In this way, we are able to train COVID-DA through standard backpropagation in an end-to-end manner.

Algorithm 1 Training of COVID-DA
Require: Labeled source data D s ={xi, yi} n s i=1 , labeled target data D t l ={xj, yj}

IV. EXPERIMENTAL RESULTS
To verify the proposed method 1 , we evaluate COVID-DA on two main aspects: (1) the performance in the diagnosis of COVID-19; (2) the algorithm characteristics of COVID-DA.
A. Experimental Settings 1) Dataset: The dataset 2 used in this experiment is collected from three open-source datasets, i.e., the COVID chest X-ray dataset [44], the COVID-19 Radiography Database 3 and the dataset of RSNA Pneumonia Detection Challenge on Kaggle 4 . Based on these collected data, we randomly choose part of normal cases and all typical pneumonia cases to make up the source domain, and use the rest of normal cases and all COVID-19 cases as the target domain. The statistics of two domains are summarized in Table I. Note that only 30% of the training COVID-19 samples are labeled in the training process, which would be more practical in real-world scenarios. Moreover, the three datasets are acquired from different countries with various imaging devices, while the tasks of two domains are similar but not completely the same. Therefore, this domain adaptation task suffers from severe domain discrepancy and task difference. In addition, as shown in Table I, the class imbalance is also severe. Considering the above issues, such a diagnosis task of COVID-19 is very challenging.
2) Compared methods: We compare COVID-DA with four categories of methods. (1) Baselines: Source-only (training the model on the well-labeled source domain), Target-only (training the model on the target domain with limited labels) and Fine-tuning (training the model on the well-labeled source domain and then fine-tuning it on the target domain with limited labels); (2) deep diagnostic models for COVID-19: DLAD [11] and COVID-Net [10] (training on the labeled target domain); (3) Unsupervised domain adaptation: MCD [31], DSN [45], DANN [30], DMAN [24], which train deep models on both the labeled source domain and unlabeled target domain; (4) Semi-supervised domain adaptation: SDT [46] and semi-DMAN (extended from DMAN [24]). Note that, the semi-supervised domain adaptation methods train deep models using both labeled source data and partiallylabeled target data.
3) Implementation details: We implement our method based on PyTorch [47]. For a fair comparison, we adopt a Resnet-18 [48] model, pretrained on ImageNet [49], as the backbone of all methods. For all compared methods, we keep the same hyper-parameters as the original paper. For COVID-DA, the details consist of two parts. (1) network architectures: We implement the feature extractor based on Resnet-18, while we implement the two domain discriminators based on two-layer fully-connected networks [24], and implement all classifiers by one fully-connected layer; (2) parameter settings: In the training process, we use an SGD optimizer with the learning rate of 0.001 to train the whole network. The batch size for each domain is set to 16. As for the trade-off parameters in Eqn. (1), we set α=0.1 and β=0.1 via crossvalidation. Following [35], we set γ=2.0 for the focal loss.

4) Evaluation metrics:
We evaluate the diagnostic performance of all considered methods for COVID-19 in terms of six metrics: F1 score (%), Recall (%), Precision (%), AUC, Sum (%) and Cost. Specifically, the first three metrics are calculated based on previous studies [24], [41], the AUC metric is based on the work [50], and the Sum and Cost metrics are based on [51], [52]. We recommend readers to these papers for detailed implementations of these metrics.   Backbone  Table II. Overall, the proposed method performs the best, which confirms its effectiveness and superiority in COVID-19 diagnosis. Since there is an urgent need for computer-aided diagnosis for COVID-19, COVID-DA makes great medical sense in practice.
According to Table II, we draw the following observations. (1) Both Target-only and Fine-tuning do not certainly outperform the Source-only on all considered evaluation metrics. One possible reason is that the deep model may overfit to the limited labeled data of the target domain, and thus perform limitedly on the test set. 2) Visualization of Grad-CAMs: In this section, we use the Gradient-weighted Class Activation Mapping [53] (Grad-CAM) method to visualize the important regions that the devised classifiers focus on for predictions. We present the visualization results of three COVID-19 patients in Fig. 2. As expected, the domain-shared classifier focuses more on the surrounding regions to capture the task-shared classification information, while the target-specific classifier focuses more on pathological regions of COVID-19 itself. By combining the above two classifiers, COVID-DA is able to make predictions based on both task-shared and target-specific classification information. In addition, the visualized Grad-CAMs are also an interpretation for the prediction of COVID-DA, which helps doctors to judge the prediction reliability in practice. 3) Ablation studies: We conduct ablation studies to evaluate the effectiveness of different components in COVID-DA. As shown in Table III, all components in our methods (i.e., feature adversarial adaptation, classifier adversarial adaptation, classifier diversity maximization, and focal loss) make empirical contributions and play important roles in our method. Particularly, domain adversarial losses (L d1 and L d2 ) are relatively important. Moreover, the classifier diversity loss (L div ) is able to further improve diagnostic performance. These results demonstrate the necessity to reduce the domain discrepancy and overcome the task difference in the task of domain adaptation for COVID-19.
C. More Discussions 1) Parameter sensitivities: As mentioned in Section IV-A3, we set trade-off parameters α = 0.1 and β = 0.1 in all experiments, where α adjusts the domain adversarial losses (L d1 and L d2 ) and β controls the classifier diversity loss (L div ). In this section, we evaluate the sensitivities of these two parameters, where we only evaluate one parameter each time, fixing all other parameters. From Table IV, our proposed method achieves the best or relatively good performance with the setting α = 0.1 and β = 0.1.
2) Domain adversarial loss: In our method, we define the domain adversarial losses (in Eqns. (2) and (3)) relying on the least square loss, since it helps to improve domain confusion and stabilize training, by preserving the domain distance information [37]. In this section, we empirically compare it with the focal loss and original GAN loss [36]. As shown in Table V, the least-square loss outperforms another two losses, which demonstrates the superiority of the adopted domain loss.
3) Classifier diversity loss: In our method, we define the classifier diversity loss (in Eqn. (4)) relying on the cosine distance. In fact, one can also use other distance metrics, e.g., L1 distance, L2 distance, KL divergence and JS divergence, according to the tasks at hand. To find the most suitable one, we conduct many preliminary experiments and report the results in Table V. To be specific, the cosine distance performs the best over all considered distances in the domain adaptation task for COVID-19.
V. CONCLUSION In this paper, we have proposed a deep domain adaptation method for the diagnosis of COVID-19 (namely COVID-DA), which aims to transfer the domain knowledge from the well-labeled source domain (i.e., typical pneumonia) to the partially-labeled target domain (i.e., . To be specific, we minimize the domain discrepancy by aligning the feature distributions of two domains via domain adversarial learning. Meanwhile, we develop a novel classifier separation scheme to overcome the issue of task difference between domains. In this way, the proposed method is able to learn a well-performed deep model with very limited annotations of COVID-19. Extensive experiments demonstrate the effectiveness and superiority of COVID-DA.
It is worth mentioning that our proposed method is of great clinical importance, since extensive annotations of COVID-19 are inaccessible now, while there is an urgent demand to develop deep learning based diagnosis methods for COVID-19.
In the future, one can apply COVID-DA to CT imaging-based diagnosis of COVID-19 and extend it to the segmentation task for fine-grained diagnosis of COVID-19.