{'fraction_non_alphanumeric': 0.03358561967833491, 'fraction_numerical': 0.014309366130558183, 'mean_word_length': 5.053686471009305, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accurate and Robust Alignment of Variable-stained Histologic Images Using a General-purpose Greedy Diffeomorphic Registration Tool

Variously stained histology slices are routinely used by pathologists to assess extracted tissue samples from various anatomical sites and determine the presence or extent of a disease. Evaluation of sequential slides is expected to enable a better understanding of the spatial arrangement and growth patterns of cells and vessels. In this paper we present a practical two-step approach based on diffeomorphic registration to align digitized sequential histopathology stained slides to each other, starting with an initial affine step followed by the estimation of a detailed deformation field.


INTRODUCTION
In the area of Digital Pathology an acceptable schema for evaluating the anatomical structure of a given dataset is to associate the visual appearance of consecutive tissue sections. Due to the way tissues can be processed, differentially stained, and other pre-analytical steps, successive slices can suffer from non-linear deformations, as well as dramatic appearance changes. This evaluation schema can be assessed by aligning consecutive slices to a common coordinate system.

MATERIALS & METHODS 2.1. Data
The Automatic Non-rigid Histological Image Registration (ANHIR) challenge [1] describes a publicly available multiinstitutional dataset [1][2][3][4][5] and a community benchmark to fairly evaluate and compare various deformable registration methods. A set of high-resolution (up to 40x magnification) whole-slide images of different anatomical sites are made publicly available. These images are organized in sets of consecutive sections of a distinct anatomical site, where each slice was stained with a different dye and any two images within a set can be meaningfully registered (Fig.1).

Pre-processing
We note that the mammary gland slides dyed for ER and c-erbB-2/HER-2-neu use a Diaminobenzidine (DAB) stain, which has a brown-dominating appearance and significant background staining that makes them very distinct from all other stained slides. Therefore, we apply a color deconvolution algorithm [6] to remove the DAB color components and avoid potential misregistrations.
Taking into consideration the large size of the scaled images used in the evaluation of our approach, we resampled them based on a factor (f) of 1/25. To prevent aliasing due to such resampling, we applied a Gaussian kernel before downsampling. All images were then converted to NIfTI format [7] for all computations.
We applied padding to each stained image to ensure that i) the sizes of paired images are the same and ii) the target tissue is in the image center. Once all image pairs are of the same size, we add an additional padding (4x the size of the similarity metric's kernel -Eq.1) to make sure that the tissue is far enough from the image boundaries, and accommodate appropriate calculation of the deformation field after changes caused by the affine registration. A binary mask excluding the image boundaries (size of the similarity metric kernel -Eq.1) is also used during affine registration. The padded areas are filled with image intensity matching that of the four corners of the unpadded image.

Registration
We register the variously-stained histopathology slices using a general-purpose 2D/3D medical image registration tool "Greedy" (github.com/pyushkevich/greedy) [8]. Greedy is a C++ implementation of the greedy diffeomorphic registration algorithm [9]. Greedy is integrated into the ITK-SNAP (itksnap.org) segmentation software [10,11], as well as the Cancer Imaging Phenomics Toolkit (CaPTkwww.cbica.upenn.edu/captk) [12,13]. Greedy shares multiple concepts and implementation strategies with the Symmetric Normalization (SyN) registration tool in the ANTS package [14,15], but focuses on computational efficiency by eschewing the symmetric property of SyN and utilizing highly optimized code for computation of image similarity metrics such as Normalized Cross-Correlation (NCC) [16].
For our approach, we use the NCC metric as: where is the scale by which the size of the fixed image is scaled so that the NCC kernel can pick up enough information about the images to perform a good registration. The registration of the image pairs is performed while considering three different scales in a configuration of a multi-resolution pyramid.
Following [17], this paper uses the notation: where ( → * ) describes the transformation between fixed ( ) and moving ( ) image, and θ defines the registration parameters yielding transformation → * . R defines a minimization process such that Eq.2 is unfolded as: where is the similarity metric, λ is a scalar parameter, and ρ is an optional regularization term. We firstly perform affine registration between the image pairs, using an optimization of the dissimilarity metric based on a Limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm (LBFGS) [18], denoted by: → = ( → ; , 0 ) (4) where 0 is the initial rigid transformation between the images. The initial transformation is obtained using a brute force search, where 5000 random rotations and translations are applied to the moving image and the combination giving the best NCC metric value is used as 0 .
Following the affine registration, we applied the diffeomorphic registration of slice j to i: Where and are the regularization parameters for the registration and N is the number of iterations required at each multi-resolution pyramid (for instance, N={100,50,10} means 100 iterations at 4x, 50 at 2x and 10 at full resolution).
Furthermore, Greedy uses an optimized smoothing of the deformation fields based on the ITK recursive Gaussian smoothing classes [19]. The actual registration is computed in an iterative manner using the update equations: → 0 = (8) where is the current iteration, → is the gradient of the metric in comparison to , is the step size, * denotes the convolution of with an isotropic Gaussian kernel with a standard deviation of and is the identity transformation. For sufficiently smaller and larger values, is smooth and has positive Jacobian determinant for all ∈ , thereby making the registration diffeomorphic in nature. As diffeomorphisms form a group under composition, → +1 is also diffeomorphic in nature [9].

Evaluation
The quantitative performance evaluation framework reported here was based on the average of the median relative Target Registration Error (rTRE). Specifically, TRE is defined as: where x T and x W are the coordinates of the landmarks in the target and warped image and de(.) defines the Euclidean distance. All TRE are then normalized by the diagonal of the image to define the rTRE: where w and h denote the image weight and height, respectively.

RESULTS
We performed a grid search for and in the range of [20,20] and we found that the optimal values based on the training data were 6 and 5 pixels, respectively.
The average across all pairs of images of the median of the rTRE for the affine and the deformable registration is equal to 0.00473 and 0.00279, respectively. Notably, our rTRE score of 0.00279 is the highest of the challenge's leaderboard.
We further note that the robustness of our method as defined by the challenge was equal to 1.

CONCLUSION
A surprising finding is that a general-purpose tool originally developed for registration of 3D medical images such as MRI, achieved excellent performance in the domain of histology registration. Greedy has previously been used for 3D reconstruction of histology and histology-MRI matching [17], and relatively little parameter tuning was needed to adopt it to this challenge. Image processing scripts will be made available through our GitHub page at github.com/CBICA/HistoReg. Future work includes the further performance evaluation of the Greedy algorithm and its comparison with alternative approaches, e.g., based on detection of salient points [20]. The overarching goal of this work is to reconstruct the 3D anatomical tissue structure from 2D histology slices [17,21,22] irrespective of the staining applied to them, in order to give more context and evaluate the association of anatomical structures in the microscopic scale with the molecular characterization of the associate tissue samples. Notably, this is of interest in cancer, where such associations are already evaluated in the macroscopic scale based on radiographic representations [23][24][25][26].