{'fraction_non_alphanumeric': 0.03511460128966354, 'fraction_numerical': 0.010151312009193642, 'mean_word_length': 4.910943396226415, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
RandStainNA: Learning Stain-Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normalization

Stain variations often decrease the generalization ability of deep learning based approaches in digital histopathology analysis. Two separate proposals, namely stain normalization (SN) and stain augmentation (SA), have been spotlighted to reduce the generalization error, where the former alleviates the stain shift across different medical centers using template image and the latter enriches the accessible stain styles by the simulation of more stain variations. However, their applications are bounded by the selection of template images and the construction of unrealistic styles. To address the problems, we unify SN and SA with a novel RandStainNA scheme, which constrains variable stain styles in a practicable range to train a stain agnostic deep learning model. The RandStainNA is applicable to stain normalization in a collection of color spaces i.e. HED, HSV, LAB. Additionally, we propose a random color space selection scheme to gain extra performance improvement. We evaluate our method by two diagnostic tasks i.e. tissue subtype classification and nuclei segmentation, with various network backbones. The performance superiority over both SA and SN yields that the proposed RandStainNA can consistently improve the generalization ability, that our models can cope with more incoming clinical datasets with unpredicted stain styles. The codes is available at https://github.com/yiqings/RandStainNA.

. The overall framework of the proposed RandStainNA.

Introduction
Pathology visually exams across a diverse range of tissue types obtained by biopsy or surgical procedure under microscopes [6]. Stains are often applied to reveal underlying patterns to increase the contrast between nuclear components and their surrounding tissues [14]. Nevertheless, the substantial variance in each staining manipulation, e.g. staining protocols, staining scanners, manufacturers, batches of staining may eventually result in a variety of hue [10]. In contrast to pathologists who have adapted themselves to these variations with years' training, deep learning (DL) methods are prone to suffer from performance degradation, with the existence of inter-center stain heterogeneity [2]. Specifically, as color is a salient feature to extract for by deep neural networks, consequently, current successful applications for whole slide images (WSIs) diagnoses are subject to their robustness to color shift among different data centers [5]. There are two primary directions to reduce the generalization error, namely stain normalization and stain augmentation [22].
Stain normalization (SN) aims to reduce the variation by aligning the staincolor distribution of source images to a target template image [11,17,24]. Empirical studies regard stain normalization as an essential prerequisite of downstream applications [2,12,22]. Yet, the capability to pinpoint a representative template image for SN relies heavily on domain prior knowledges. Moreover, in real-world settings such as federated learning, the template-image selection is not feasible due to the privacy regularizations [10], as source images are inaccessible to the central processor as a rule. Some generative adversarial networks (GANs) are proposed recently [17,18] for SN, yet remaining the phenotype recognizability is always problematic. A salient drawback of the sole stain style in SN is the restricted color-correlated features can be mined by deep neural networks.
. The three-step training stage comprises a random selection of color space S, the generated of an associated random stain style template , and the normalization of a batch with the generated virtual template. Our approach is downstream task agnostic.
Stain augmentation (SA) seeks a converse direction to SN by simulating stain variations while preserving morphological features intact [21]. Tellz et. al. [20] first tailored data augmentations from RGB color space to H&E color space. Afterward, in parallel with SN approaches, GAN is also widely adopted by stain augmentation applications e.g. HistAuGAN [23].
Previous works have compared the performance between SN and SA without interpretation of their differences [22]. Moreover, we have observed that the mathematical formulations of SN are coincidental with SA, where the transfer of SN depends on a prior Dirichlet distribution [25] and SA distorts images with a uniform distribution [20], depicted in Fig. 1. Hence, we make the first attempt to unify SN and SA for histology image analysis. Two primary contributions are summarized. First, a novel Random Stain Normalization and Augmentation (RandStainNA) method is proposed to bridge stain normalization and stain augmentation, consequently, images can be augmented with more realistic stain styles. Second, a random color space selection scheme is introduced to extend the target scope to various color spaces including HED, HSV, and LAB, to increase flexibility and produce an extra augmentation. The evaluation tasks include tissue classification and nuclei segmentation, and both show our method can consistently improve the performance with a variety of network architectures.
Method Overview. Random Stain Normalization and Augmentation (Rand-StainNA) is a hybrid framework designed to fuse stain normalization and stain augmentation to generate more realistic stain variations. It incorporates randomness to SN by automatically sorting out a random virtual template from pre-estimated stain style distributions. More specifically, from the perception of SN's viewpoint, stain styles 'visible' to the deep neural network are enriched in the training stage. Meanwhile, from the perception from the SA's viewpoint, RandStainNA imposes a restriction on the distortion range and consequently, only a constrained practicable range is 'visible' to CNN. The framework is a general strategy and task agonist, as depicted in Fig. 2.
Stain Style Creation and Characterization. Unlike the formulation of comprehensive color styles of nature images, stain style of histology remains to be a vague concept, which is primarily based on visual examination, restricting to obtain a precise objective for alleviating the stain style variation [16,24]. To narrow this gap, our work first qualitatively defines the stain style covering six parameter, namely the average and standard deviation of each channel in LAB color space [16]. We pick up LAB space for its notable capability to represent heterogeneous styles in medical images [16]. Novelly, we first transfer all histology slides in the training set from RGB space to LAB color space. Then the stain style of im- are the average value and standard deviation of each channel c ∈ {l, a, b} in image x i , as shown in the pre-processing stage block in Fig. 2.
Virtual Stain Normalization Template. In routine stain normalization approaches [16,24], a source image is normalized to a pre-selected template image by aligning the average of A s and standard deviation D s of pixel values to the template A t and D t . Thus, it is sufficient to formulate a template image with [A t , D t ]. In the proposed RandStainNA, we expand the uniformly shared one-template mode to a board randomly generated virtual templates [A v , D v ] scheme. To be more specific, iteratively, random A v is sampled from distribution F A , and likewise D v is picked out from the other distribution F D , which are jointly used as the target for every training sample normalization. Empirical results yield that eventual performance are robust to a wide range of distribution types of F A and F D , such as Gaussian and t-distribution. In the rest of this section, we simply leverage Gaussian distribution as the estimation i.e. setting writes for the Gaussian distribution with expectation M and covariance matrix Σ. Notably, due to the orthogonality of channels, Σ is a diagonal matrix i.e. Σ = diag(σ 2 1 , σ 2 2 , σ 2 3 ) for some σ j with j = 1, 2, 3.

Statistics Parameters Estimation For Virtual Template Generation.
The estimation of statistical parameters of M A , Σ A , M D , Σ D are afterwards applied to the formation of stain style discussed above. A proper candidate is attributed to the sample channel mean values of all the training images for M A and M D , as well as the standard deviations of samples for Σ A and Σ D , based on the average value and standard deviation of the whole training set. However, two defects turn out in this discipline that one is the inefficiency to transverse the whole set, and the other is the special cases of infeasibility e.g. federated learning or lifelong learning. Therefore, we provide a more computation-efficient alternative, by randomly curating a small number of patches from the training set and applying their sample mean and standard deviation as The empirical results suggest it can achieve competitive performance.
Image-wise Normalization With Random Virtual Template. After transferring image x i from RGB into LAB space, we write the pixel value as [l, a, b]. We denote the average and standard deviation (std) for each channel of this image as i ], and the generated random virtual template associated to Then the image-wise normalization based on random virtual template is formulated as Then, we transfer [l , a , b ] from LAB back to RGB space. Notably, we generate different virtual templates for images that vary at every epoch during the training stage. Therefore, RandStainNA can largely increase the data variations with the on-the-fly generate virtual templates.
Determine Random Color Space for Augmentation. By the computation of the stain style parameters [M, Σ] of distinct color spaces, we can derive their associated F A and F D . Afterwards, we extend our RandStainNA from LAB to other color spaces e.g. HED, HSV. This extension allows the proposal of a random color space selection scheme, which will further strengthen the regularization effect. The candidate pool comprises three widely-used color spaces in the domain of histology, i.e. HED, HSV, LAB. Training iteratively, an initial color space S is an arbitrary decision with equal probability i.e. p = 1 3 , or with manually-assigned values depending of the performance of each independent space. Subsequently, a virtual template is assigned to associate with S to perform image-wise stain normalization.

Experiments
Dataset and Evaluation Metrics. We evaluate our proposed RandStainNA on two image analysis tasks i.e. classification and segmentation. Regarding the patch-level classification task, we use a widely-used histology public dataset NCT-CRC-HE-100K-NONORM for training and validation, with the addition of the CRC-VAL-HE-7K dataset for external testing [9]. These two sets comprise a number of 100,000 and 7180 histological patches respectively, from colorectal cancer patients of multiple data centers with heterogeneous stain styles. We randomly pick up 80% from NCT-CRC-HE-100K-NONORM for training and the rest 20% for validation. The original dataset covers nine categories, but for the category of background, we can straightforwardly identify them in pre-processing stage with OTSU algorithm [15] and thus it is removed in our experiment for a more reliable result. The top-1 classification accuracy (%) is used as the metric for the 8-category classification task. For the nuclei segmentation task, we use a small public dataset of MoNuSeg [12], with Dice and IoU as the metrics.
Network Architecture and Settings. In the classification task, we employ six backbone architectures to perform the evaluations, namely the ResNet-18 [7], ResNet-50 [7], MobileNetV3-Small [8], EfficientNetB0 [19], ViT-Tiny [3] and SwinTransformer-Tiny [13]. These networks, including CNN and transformers, may represent a wide range of network capabilities, which effectively demonstrate the adaptability of our method in different settings. In the nuclei segmentation task, we use CIA-Net as the backbone [26] for its notable performance in small set processing. We use a consistent training scheme for distinct networks for performance comparison with stain augmentation and stain normalization methods. Detailed training schemes and hyper-parameter settings are shown in the supplementary material. We perform 3 random runs and compute the average for each experiments.
Compared Methods. All models are trained with morphology augmentation, namely the random vertical and horizontal flip. In both evaluation tasks, we compare our method with existing stain normalization [16] and stain augmentation [21,22] approaches performed in the three color spaces i.e. HED, HSV, LAB. Regarding the stain augmentation in HED, we employ a multiplication rule [22] that adds noise to each channel i.e. p = p * ε 1 + ε 2 , where p is the augmented pixel value and p is the original pixel value, and ε 1 and ε 2 are uniform random noises, termed as stain augmentation scheme #1 (SA1). For the SA in HSV, we adopt an addition rule i.e. p = p + p * ε [22], termed as stain augmentation scheme #2 (SA2). We integrate the above two schemes for LAB stain augmentation, due to an absence of literature works for SA in LAB. We also configure two augmentation settings according to different degrees of distortion i.e. range of random noise, denoted as light (L) and strong (S) [21,22]. To fully retain recognizable morphological features, we do not take GAN-related approaches for comparison. Table 1. Test accuracy (%) comparison on the tissue type classification task. We compare our method with stain augmentation (SA) [22] and stain normalization (SN) [16] in three color space i.e. LAB, HSV, HED [22]. In SA, we follow previous work [22] by leveraging two settings, namely light (L) and strong (S), determined by the degree of distortion. The best and second best are marked in boldface and with * respectively.

Method
ResNet18 Results. Our method can consistently improve the baseline performance of the six backbone architectures in terms of test accuracy, with the implementation in three color spaces, as shown in Table 3. Therefore, it yields the effectiveness of RandStainNA. The hybrid architecture can outperform a sole deployment of either SN or SA. With the random color space selection scheme (denoted as 'full'), the RandStainNA achieves further performance improvement. To demonstrate the effects achieved with different approaches straightforwardly, we visualize the original raw images with stain variations, SN images, SA images, and images processed with our RandStainNA in Fig. 3. In the visualization graph, we use the results from SA and SN performed in HSV space as an example, which shows very similar outcomes in LAB and HED spaces. As shown, the SN unifies stain styles into a shared template that may leave out many useful features [22], and the SA may generate unrealistic images. In contrast, our method generates much more realistic images to reorganize by both human and deep learning techniques. The  ble 2. For SN and SA in each color space, we pick up one configuration with higher performance in the classification task. Our method also achieves the best performance to demonstrate its effectiveness in various downstream tasks.

Supplementary Material
Step decay 0.95 per ep  Table 6. Patch-level classification performance comparison on BACH in terms of test top-1 accuracy and AUC. The Data, annotations, and description of BACH is freely available at https://iciar2018-challenge.grand-challenge.org/. For all the compared methods, we follow the same settings as Table 2 in the paper. We use ResNet-18 as the backbone.