{'fraction_non_alphanumeric': 0.03059483178937104, 'fraction_numerical': 0.01029985372988786, 'mean_word_length': 4.889806173725772, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Segmentation of HER2 protein overexpression in immunohistochemically stained breast cancer images using Support Vector Machines

Breast cancer is one of the most common cancers in women worldwide. Patient therapy is widely supported by analysis of immunohistochemically (IHC) stained tissue sections. In particular, the analysis of HER2 overexpression by immunohistochemistry helps to determine when patients are suitable to HER2-targeted treatment. Computational HER2 overexpression analysis is still an open problem and a challenging task principally because of the variability of immunohistochemistry tissue samples and the subjectivity of the specialists to assess the samples. In addition, the immunohistochemistry process can produce diverse artifacts that difficult the HER2 overexpression assessment. In this paper we study the segmentation of HER2 overexpression in IHC stained breast cancer tissue images using a support vector machine (SVM) classifier. We asses the SVM performance using diverse color and texture pixel-level features including the RGB, CMYK, HSV, CIE L*a*b* color spaces, color deconvolution filter and Haralick features. We measure classification performance for three datasets containing a total of 153 IHC images that were previously labeled by a pathologist.


Introduction
A common practice for evaluating the progression of diverse types of cancers is the analysis of immunohistochemistry-stained tissue sections. Immunohistochemistry (IHC) is a method for detecting specific antigens in tissues or cells based on antigen-antibody recognition. This technique takes advantage of the specificity provided by the binding of an antibody with its antigen shown at a microscopic level [1]. The antigen-antibody interaction is visualized through a color-producing reaction in various cellular structures such as nucleus, cytoplasm or membrane, for instance.
A typical biomarker observed on the cell membrane is the Human Epidermal growth factor Receptor 2 (HER2). This is a protein normally expressed on the cell membrane surface of various organs epithelia such as lung, bladder, pancreas, breast, stomach and prostate [2]. When HER2 is present in normal amounts, the protein plays an important role in normal cell growth development. However, when the HER2 protein is overexpressed, cells multiply and grow more rapidly than normal, promoting carcinogenesis in diverse types of cancer [3].
The HER2 protein has become an important biomarker for invasive breast cancer. Around 15%-25% of breast cancer overexpress HER2 protein. Its measurement helps to guide the therapeutic decision process. When IHC is used to assess the amount of HER2 overexpression, two main chemical compounds (commonly referred to as stains) are used to visualize cell nuclei and membranes: hematoxylin and diaminobenzidine (DAB). The hematoxylin is used as a counterstain and produces a blue-violet staining of nuclear elements. The diaminobenzidine produces a dark brown reaction on the cell membrane (only when there is HER2 overexpression).
In general, pathologists perform a qualitative assessment of HER2 overexpression through a visual inspection and classify IHC tissues according to the ASCO/CAP guidelines [4]. They should estimate two main parameters associated to HER2 overexpression: intensity and completeness. The former corresponds to the intensity of brown staining (from weak to intense) and the latter corresponds to the percentage of staining completeness with respect to the membrane length.
According to the two estimated parameters the observed specimen is classified into four categories: 0, 1+, 2+ and 3+. The classification system proposed in [4] indicates: 0 means no staining was observed, 1+ barely perceptible, 2+ weak to moderate and 3+ means strong intensity was observed in more than 10% of tumor cells (see Fig. 1).
Thus, the classification (and the whole analysis) is mainly based on the strength of the staining, whose evaluation clearly depends on the experience of the pathologist. This subjective analysis can produce inconsistent results. Hence, the importance of a quantitative and objective analysis is recognized by researchers in both, the image analysis and pathology fields [5].
Several cell segmentation methods have been proposed. However HER2 overexpression segmentation is still and open problem. This is due to the non-standardization of tissue samples and also due to the undesired brown staining over the cytoplasm. Regarding nonstandardization, immunohistochemical tissue samples can be quite variable due to the use of different whole slide imaging (WSI) scanners, immunohistochemical kit systems or staining procedures [6], [7], [8]. This tissue variability makes difficult the finding of a general tissue pattern. On the other hand, HER2 overexpression is visualized through the brown color reaction produced on the membrane, but it is quite frequent that brown color spreads over the cytoplasm, which is an artefact and should not be considered as HER2 overexpression.
In this study we address the problem of segmentation of cell membranes with HER2 overexpression. This is a previous but crucial step before being able to estimate intensity and completeness parameters of the HER2 overexpression. We proposed a method for the segmentation of HER2 overexpression using a support vector machine (SVM). We asses the SVM performance using diverse color and texture pixel-level features including the RGB, CMYK, HSV, CIE L*a*b* color spaces, color deconvolution filter and Haralick features.
The outline of this paper is described as follows. Section 2 describes the proposed method, including the computation of color and texture features and SVM pixel classification. In section 3 we measure the performance of SVM classifiers on three different set of features and three datasets including images 1+ in data set one, images 2+ in data set two and images 3+ in data set three. Finally, Section 4 presents the conclusions and perspectives of this work.

Segmentation using SVM
The proposed method addresses the segmentation of HER2 overexpression as a binary classification problem and each pixel belongs to one of the two classes: overexpressed and nonoverexpressed. The method uses a SVM as binary pixel classifier. As mentioned before, IHC images can be highly variable which makes difficult to find a common tissue pattern. Color variation is one of the most important issues, and this is produced for a number of reasons such as: the variable histochemical stains kits coming from different manufacturers, the staining procedure (which in many cases is performed in a manual way), or the use of different whole slide imaging (WSI) scanners. In fact, there is a lack of standardization between IHC clinical laboratories. Current developments in this field strongly suggest the need for the standardization of this process [9,8,7,6]. Thus, considering this variable scenario, we decide to build one SVM classifier per IHC image.
The proposed method includes three main stages: pixel-level feature computation, training data generation, and SVM pixel classification (see Fig. 2). The aim of feature computation is to represent the IHC image pixels by means of a descriptive set of features that captures the complex characteristics of HER2 overexpression. In this paper, we compute a set of 210 color and texture features. Training data generation is a fundamental step of any supervised machine learning method. In this study, training pixels are selected from a small sub-image of the IHC image. Then, we build the SVM classifier in order to predict the label of the remaining pixels. Notice that the selection of the sub-image is an important issue. The proposed method needs that the expert provides training data to build the SVM classifier. This training data should be a small but representative set of the whole IHC image. The point of doing this is that we want to include the pathologist knowledge in order to guide the segmentation, due to the complexity of the IHC images, and pathologist are able to label small portions of the whole IHC-image. In this paper, we have the gold standard of the whole IHC image, thus, we are able to simulate when the pathologist generate the training data.

Pixel-level Feature Computation
In this work, we calculate a total of 210 color and texture pixel-level features (see Table 1). We use color features that correspond to different space color channels and a color deconvolution filter. We use four color spaces: RGB, CMYK, HSV and CIEL*a*b*, together with the grayscale image. In addition, we use a color deconvolution filter for stain separation in IHC stained images. This deconvolution is achieved by means of a transformation of the original RGB IHC image, which is based on the Lambert-Beer law and the optical density. In particular, we use the DAB channel obtained with the color deconvolution filter [10].
Texture features includes the calculation of Gray Level Co-occurrence Matrices (GLCMs) in order to obtain GLCM mean, GLCM standard deviation and eleven Haralick texture features [11] (a total of thirteen features). This calculation has a high computational cost, because we are computing GLCM matrices in a pixel-level. For this calculation we define a neighborhood for each pixel and a GLCM matrix is calculated for each neighborhood. Then, from each GLCM matrix, we calculate thirteen texture features for each pixel. Due to the nature of the algorithmic process, we implemented a set of parallel routines in C++ code, using the MPI library, in order to speed up the whole process.

SVM pixel classification
Support Vector Machines (SVM) are supervised learning algorithms widely used for binary classification. SVM were introduced by Vladimir Vapnik et al. [12] and the general idea is to find an optimal separating hyperplane that maximizes the minimum distance between any data point in order to separate data into two classes. We use SVM as a binary pixel classifier, thus, the segmentation of HER2 overexpression is handled as a classification problem. Each pixel is classified into one of the following classes: overexpressed or non-overexpressed. We build a SVM model for each IHC image using the pixels of a sub-image (training data). Then we predict the class labels of the remaining pixels (testing data).

Data
A total of 153 images were used for the experiments, including around 750 cells. The images are grouped into three datasets. Dataset 1, dataset 2 and dataset 3 contain images of category 1+, 2+ and 3+, respectively. The images are regions of interest (ROIs) extracted from paraffin embedded breast cancer tissue slides obtained from the archives of the Department of Pathology at the Hospital Clínico Universidad de Chile. The tissues were scanned using the NanoZoomer-XR C12000 whole slide imaging scanner (Hamamatsu Photonics K.K.) at a magnification of 40x and a resolution of 0.23 µm/pixel. All images were manually labeled by a pathologist in order to generate training data and gold standard for the evaluation of the proposed method.

Experiment Setup
The goal of this experiment is to analyze the SVM performance using three set of features: the whole set of 210 features; (ii) a subset of 4 features corresponding to the channels of CMYK color space; (iii) and another subset of 14 features corresponding to the texture features applied to the gray channel (see Table 1). For building the SVM classifieres, we select the Gaussian Radial Basis Function (RBF) as the SVM kernel function and we performe 5-fold cross-validation. The F1 metric [13] is used to measure performance in the cross-validation stage. We select the F1 metric because we are dealing with an unbalanced classification problem (there are more pixels belonging to the nonoverexpressed class than the overexpressed class) [14]. Notice that the most commonly metric used in the machine learning literature is accuracy [13], however it is not well suited in this context.
In this experiment, for each IHC image I, we generate N sub-images s k , k = 1, . . . , N . All the pixels of the sub-image s k are used to train SVM classifiers svm k , k = 1, . . . , N . For each k, the svm k classifier is then used to classify the remaining pixels of the IHC image. Since we have the gold standard of the IHC image, we can calculate the performance η k of the svm k classifier. From the N sub-images s k , we select the sub-image which gives the best SVM performance. We call this sub-image the "best sub-image", s * (see Fig. 3).
This selection is repetead for each IHC image of our dataset. The performance presented in this paper correspond to the performance of the best sub-images of each IHC image. In addition, we group them into the 1+,2+ and 3+ categories and into the three different set of features (see box plots of Fig. 4).
Regarding the computational resources, we used a DELL Studio server to run the experiments. This server has 8 cores (Intel(R) Core(TM) i7 CPU 960 @3.20GHz) 24 GB RAM, and the Archlinux operating system. The framework was mainly programmed in Python, using the scikit-learn library [15] for machine learning algorithms. In particular, the Haralick features were calculated in the datura cluster (Max-Planck-Institut für Gravitationsphysik, Albert-Einstein-Institut, Germany), which has a total of 165 nodes (we used 128), and each node has 12 cores (CPU: Intel(R) Xeon(R) CPU X5650 @ 2.67GHz), 24 GB RAM, Scientific Linux 6.0 operating system, and InfiniBand for nodes communication.

Performance Evaluation
The SVM pixel classification performance is evaluated using five different metrics: ROC AUC, accuracy, precision, recall, and F1 [13]. The results can be seen in the box plots of Figure 4 and  Figure 3. Selection of the best sub-image. We divide the image into N sub-images (N = 9 in this example) and we rank the sub-images according to the performance that is measured on the testing data (the whole IHC without the training data). The sub-image with the best performance, is called the best sub-image.
a summary of these is shown in Table 2.
In most of the cases, the CMYK feature set produced the better SVM performance. We may expect that using the full feature set (210 features) should produce the best performance, but the experiments with high dimensionality do not show this trend. Recall that the performance is measured using a gold standard manually labeled by one expert pathologist. This is a critical point, because the intrinsic subjectivity and variability of the gold standard can affect our results. Ficarra et al [16] clearly describe this situation.
In addition, the experiments show that the best sub-image is not always the same. We observed that when we change the feature set and the performance metric, the best sub-image also changes.

Conclusions
We have presented a method to segment HER2 overexpression in IHC stained breast cancer tissue images. We have described the main stages of the method, including the computation of 210 color and texture features, training data generation and SVM pixel classification; and performed experiments to evaluate the contribution of different sets of features to the SVM performance.
The experimental results have shown that using the best sub-image for training data selection and increasing the dimensionality of the feature set do not necessarily produce a better performance (considering our SVM configuration). The results showed that the set of 4 features corresponding to the CMYK color space produced the better SVM performace in most of the cases. This suggests that the aggregation of additional features, i.e., increasing dimensionality, does not contribute to a better segmentation. However, this is not enough evidence to remove those extra dimensions.
Two important aspects of our study have to be highlighted. First, we used a gold standard that was manually labeled by a pathologist. Thus, the interpretation of our results must consider the intrinsic subjectivity of the gold standard. Second, we were able to rank the sub-images because we have a gold standard, however this is precisely what is missing for a real case. Therefore, our next goals will be to continue analyzing the best set of features (using a gold standard generated for more than one pathologist), and developing methods able to detect the best sub-image, but now, from unlabeled set of images.