{'fraction_non_alphanumeric': 0.03404867256637168, 'fraction_numerical': 0.013008849557522123, 'mean_word_length': 5.311225914549008, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatic discovery of clinically interpretable imaging biomarkers for Mycobacterium tuberculosis supersusceptibility using deep learning

Background Identifying which individuals will develop tuberculosis (TB) remains an unresolved problem due to few animal models and computational approaches that effectively address its heterogeneity. To meet these shortcomings, we show that Diversity Outbred (DO) mice reflect human-like genetic diversity and develop human-like lung granulomas when infected with Mycobacterium tuberculosis (M.tb) . Methods Following M.tb infection, a “supersusceptible” phenotype develops in approximately one-third of DO mice characterized by rapid morbidity and mortality within 8 weeks. These supersusceptible DO mice develop lung granulomas patterns akin to humans. This led us to utilize deep learning to identify supersusceptibility from hematoxylin & eosin (H&E) lung tissue sections utilizing only clinical outcomes (supersusceptible or not-supersusceptible) as labels. Findings The proposed machine learning model diagnosed supersusceptibility with high accuracy (91.50 ± 4.68%) compared to two expert pathologists using H&E stained lung sections (94.95% and 94.58%). Two non-experts used the imaging biomarker to diagnose supersusceptibility with high accuracy (88.25% and 87.95%) and agreement (96.00%). A board-certified veterinary pathologist (GB) examined the imaging biomarker and determined the model was making diagnostic decisions using a form of granuloma necrosis (karyorrhectic and pyknotic nuclear debris). This was corroborated by one other board-certified veterinary pathologist. Finally, the imaging biomarker was quantified, providing a novel means to convert visual patterns within granulomas to data suitable for statistical analyses. Implications Overall, our results have translatable implication to improve our understanding of TB and also to the broader field of computational pathology in which clinical outcomes alone can drive automatic identification of interpretable imaging biomarkers, knowledge discovery, and validation of existing clinical biomarkers. Funding National Institutes of Health and American Lung Association.


Introduction
Tuberculosis (TB) is an important global disease. Over 2 billion people are currently infected with Mycobacterium tuberculosis (M.tb), the bacterium that causes TB. In 2018, 10 million people were diagnosed with TB and 1.5 million people died, surpassing mortality due to HIV/AIDS [1]. More than 4000 deaths occur per day due to TB. When infected with M.tb, those susceptible develop lung disease, called active pulmonary TB. Clinical symptoms are fever, cough, progressive weight loss and emaciation, due to lung inflammation, necrosis, and cavitation. Death occurs in 40-70% of untreated cases [2,3]. Fortunately, most (90%) humans are resistant to M.tb and, if infected, survive with latent M.tb infection (LTBI) for decades. Although a small fraction of LTBI cases transition to active TB due to acquired immunodeficiency, diabetes, and old age [4], most patients who progress to active pulmonary TB have no known risk factors. Additional forms of TB are recognized clinically (e.g. fulminant, miliary, subclinical, and incipient) and are part of the spectrum of human responses to M.tb. There are no consensus lung or blood biomarkers readily available to diagnose active pulmonary TB, and examination of lung tissue in sick patients is performed by in-life scans, after surgical removal or at autopsy. As few animal models develop humanlike lung granulomas, there is a need for additional animal models of TB to improve translational relevance of experimental findings.
To address needs for a better mouse model of TB, we use Diversity Outbred (DO) mice. Each DO mouse's genome is a heterozygous mosaic of DNA inherited from the 8 founder strains (Supplemental Fig. 1). The population was created by breeding together 5 Mus musculus spp domesticus strains (A/J; C57BL/6J; 129S1/SvlmJ; NOD/ShiLtJ; NZO/HILtJ) and 3 wild-derived Mus musculus strains (CAST/EiJ; PWK/ PhJ; WSB/EiJ) [5,6]. The genetic diversity of the DO population rivals the genetic diversity of the human population and this has been exploited to understand the genetic basis of disease [6À14]. When infected virulent M.tb, DO responses better emulate human forms of TB than C57BL/6, BALB/c, CBA/J, or C3HFeB/HeJ inbred strains [2,15À19]. Analogous forms of TB in humans and DO mice are shown below (Table 1).
Following infection with a low dose of aerosolized M.tb bacilli, DO mice develop wide phenotype ranges in survival, weight change, lung granuloma morphotypes, acquired immunity, and innate inflammatory responses [20À22] that are not observed in inbred strains including the related Collaborative Cross recombinant inbred lines [5,6,19,22À30]. Like humans, weight loss in M.tb-infected DO mice reflects metabolic signatures of poorly regulated inflammation (unpublished). Also like humans, biomarker signatures from lungs and/or serum better discriminate TB disease forms in DO mice than single biomarkers [23]. Since those initial studies, we are now generating larger data sets to find accurate protein biomarkers for diagnosis and to generate testable mechanistic hypothesis. Finally, we have applied image analysis and deep learning models to automatically detect histologic features of M.tb-infected lungs, including granulomas, cell-poor caseous necrosis, lymphocytic cuffs, macrophage-rich regions, neutrophil-rich regions, normal lung tissue, and acid-fast stained M.tb [21,23,31À33].
Our extensive deep learning methods to automatically extract information from lungs of M.tb-infected DO mice are limited by three key factors, reflected in digital pathology more generally. First, the vast majority of deep learning models require strong labels [34]. Strong labels can be thought of as any label that can be delineated in an image (such as nuclei or tissue layers), whereas weak labels can describe an image more generally (e.g. a morphological diagnosis applied to a slide). Acquisition of strong labels or manual annotations for digital histology images is a barrier for deep learning in computational pathology [35]. In our problem À diagnosis of supersusceptibility to M.tb À manual annotation for strong labels is not an option, as there are no consensus histopathological features or specific cell types indicative of supersusceptibility. Second, deep learning models do not efficiently process the amount of data in digital histology images, on the order trillions of pixels. In other domains, large images can be resized [36À41]. However, in computational pathology, resizing results in information loss including individual cells, locations, and tissue-level microanatomy (like looking at low-magnification). As an option to resizing, tiles (small images cropped from a large digital images) are often substituted for the whole image, but this approach still requires manual annotations (i.e. strong labels) and is computationally expensive to process hundreds of thousands of tiles per tissue section. Third, how deep learning models make decisions for a histology side is not interpretable by humans and therefore difficult to trust. This 'black-box' nature of computational pathology limits acceptance in biomedical research and medicine, as both scientists and clinicians wish to know how decisions are made before the information is used to inform a biological mechanism or to make an informed clinical decision. Tools such as class activation mapping (CAM) [42], Grad-CAM [43], and Grad-CAM++ [44] can highlight which parts of an image contribute to what the deep learning model "sees" but cannot be applied in computational pathology for diagnosis, as digitized tissue sections are too large.
To overcome these limitations, we implement an attention-based multiple instance learning (MIL) [45] model to identify supersusceptible DO mice using hematoxylin and eosin (H&E) stained lung tissue

Research in context
Evidence before this study We performed literature searches for publications in English without date restrictions. We searched PubMed for "tuberculosis AND (mouse model) AND (granuloma necrosis)" on April 27th 2020. This retrieved 116 publications: 15 reviews and 101 primary research articles. Twenty-six primary articles reported granuloma necrosis: 24 with inbred or gene-deleted mice, and 2 with humanized mice or inbred mice with human transgenes. No primary publications except our own used Diversity Outbred mice. No primary research articles applied artificial intelligence to H&E lung granulomas. We searched PubMed for (tuberculosis AND (''imaging biomarker'') on April 27th 2020. This retrieved 3 publications, all on PET or CT scans of human TB patients and none on histology images. We searched PubMed and arXiv for the term "(multiple instance learning) AND (histology OR pathology OR histopathology)" on September 18, 2019 without date restriction and limited to English articles. This resulted in a total of 35 articles from PubMed and 24 articles from arXiv. Six of these results applied deep multiple instance learning to predict some diagnostic label using histopathology images. All studies evaluated their methodology on cancer histology slides, including colon, prostate, basal cell, and breast. Five studies used weak labels for training of their models, and one required regions of interest. Five studies applied their method to a two-class problem. Four used an attentionbased mechanism for pooling multiple instances into a slidelevel decision. All studies mentioned interpretability, but only one study emphasized this and verified it with a pathologist. Further, no study compared model performance to pathologist performance or non-expert performance, verified model-identified key instances with current clinical practices, quantified model-identified key instances, or recognized the potential for their model to discover novel imaging biomarkers (in which tissue-level annotations are impossible).

Added value of this study
Attention-based deep learning can automatically discover clinically relevant histopathology-based biomarkers and can quantify these features. The latter (quantification) is impossible for human pathologists to perform and is a major strength of this study as visual information can be extracted and quantified for statistical analyses. Further, in supersusceptibility to M.tb infection, both our model and non-experts exceeded pathologist performance using model-identified imaging biomarkers.

Implications of all the available evidence
Deep learning in pathology is stifled by its requirement for meticulously curated ground truth, inefficiency in processing large images (like in histopathology), and its "black-box" nature. Our experiments with M.tb-infected mice implicate a model that automatically identifies interpretable imaging biomarkers for disease using only a diagnostic label. Not only does our proposed machine learning model meet these shortcomings of deep learning, it also more broadly paves a path for the discovery of new interpretable imaging biomarkers and validation of current practices in clinical pathology.
sections. Unlike conventional deep learning in computational pathology, the proposed machine learning model requires only weak labels, can efficiently process the large digital histopathology images without resizing, is interpretable, and automatically identifies regions in H&E slides that contribute to its clinical label. Although similar contemporary studies recognize these aspect of attention [46À50], only one study verified biological interpretability with an expert pathologist [51]. Further, no studies compare model performance to pathologist performance or non-expert performance, verify model-identified key instances with current practices, or recognize the potential for their model to discover novel imaging biomarkers. As evidenced by this study, the field of computational pathology may benefit from the proposed machine learning model's interpretability, as it provides a mechanism for the automatic discovery of novel image biomarkers as well as validation of current practices.

Ethics statement
All procedures were approved by Tufts University's Institutional Animal Care and Use Committee, and the Institutional Biosafety Committee. These experiments were approved under IACUC protocols: G2012-53; G2015-33; G2018-33. Biosafety Level 3 (BSL3) work was approved by under IBC registrations GRIA04; GRIA10; and GRIA17.

Mice and M.tb infection
Female DO mice (n = 452) and female C57BL/6J inbred mice (n = 30) from The Jackson Laboratory (Bar Harbor, ME) and were housed with sterile caging, bedding, food, and water in the biosafety level 3 (BSL3) facility at the New England Regional Biosafety Laboratory (Tufts University, Cummings School of Veterinary Medicine, North Grafton, MA). At 8-10-weeks old, mice were infected with low (~100 bacilli, n = 176) or very low (~20 bacilli, n = 256) dose of M.tb strain Erdman bacilli using a CH Technologies nose-only system, as we have described [23]. Mice were randomly assigned to cages prior to infection. Mice were monitored for health daily, weighed thrice weekly, and euthanized when signs of morbidity due to pulmonary TB developed (i.e. loss of body condition, respiratory distress). All mice were confirmed to be infected with M.tb. Age and gender matched non-infected control DO mice (n = 40) were identically housed, monitored, and euthanized at experimental end points. We selected a DO sample size to provide sufficient power to detect a quantitative trait locus that accounts for 10% of the phenotypic variance with 80% power at an alpha of 0.5 based on power simulations presented in [13]. We also provided enough mice to have at least 140 mice in each of the susceptibility classes.

Genome construction of DO mice
DO haplotypes are constructed using allele calls from GigaMUGA mouse genotyping array performed by Neogen (Lincoln, NB) and a hidden Markov model (HMM) implemented in R software as described [9].

Diagnostic categories for M.tb infected mice
The "supersusceptible" and "not-supersusceptible" ground truth labels reflect clinical outcomes that occurred during experimental M. tb infection. Supersusceptible DO mice developed morbidity and mortality within 8 weeks of M.tb infection. Mice that survived 8 weeks without morbidity or mortality were "not-supersusceptible." These two phenotypes are robust and reproducible, as we have previously observed [21,22]. Mice were excluded if they were euthanized due to non-TB disease based on clinical examination and necropsy findings. The result total number of supersusceptible mice was 148, and the total number of not-supersusceptible mice was 266.

Slide preparation and digital images
After euthanasia, lungs from each mouse were inflated and fixed in 10% neutral buffered formalin, processed and embedded in paraffin, sectioned at 5mm and stained with hematoxylin & eosin (H&E) at Tufts University, Cummings School of Veterinary Medicine, Core Histology Laboratory (North Grafton, MA). H&E stained glass slides were magnified 400 times and digitally scanned by Aperio ScanScope at 0.23 microns per pixel (The Ohio State University's Comparative Pathology and Mouse Phenotyping Shared Resources Core Facility, Columbus, OH). The median image size was 153,384 £ 82,575 pixels. A flow diagram for the study design can be found in Fig. 1a.

Multiple Instance Learning (MIL)
MIL is a machine learning method where weak labels are assigned to collections (called bags) rather than individual examples (called instances) like in conventional machine learning. Classification by MIL is performed at the bag level and not the single instance level like in supervised learning [52]. The underlying assumption is that one class ("positive") shares features with a second class ("negative") and possesses features unique to itself. Here, the "bags" are the whole slides with one of two possible ground truth class labels À supersusceptible and not-supersusceptible. The "instances" are the unannotated, small images sampled from H&E-stained lung sections. The MIL paradigm applies because lungs from M.tb-infected supersusceptible DO mice share microscopic features with lungs of mice notsupersusceptible (e.g., regions of normal lung tissue, lymphocytes, plasma cells, macrophages) and also contain unique features (e.g. large necrotizing granulomas infiltrated by many neutrophils).

Attention-based pooling
The attention-based pooling mechanism [45] automatically learns to dynamically weight embedded instances into a bag-level feature vector that is subsequently classified. For example, if we have a single mouse, we take images sampled (instances) from its digital H&E tissue section (bag) and extract features from each, forming instance embeddings. A weight is automatically computed for each embedded instance through the attention-based pooling mechanism, then a weighted sum combines them into a single, bag-level instance, corresponding slide-level embedding. Classification is then performed on this bag-level embedding.
Our attention mechanism implementation consists of a simple two-layer fully connected network which passes each instance embedding (h k ) though one layer of the network (V), applies a tanh activation function to the result, then passes the activation though the second layer (w T ), which maps the vector into a single value, its attention weight (a k ). The weighted sum of each embedded instance and its attention weight yields a bag-level instance (z). The parameters (V,w) for this for this two-layer neural network are automatically learned through training of the model.
In addition to performing better than instance-based and embedding-based max and mean pooling approaches, the resulting instance weights allow the model to be interpretable in that the relative magnitudes of instance weights directly correspond to the instance's relative contribution to the overall classification of the bag [45]. Practically speaking, this means the model automatically identifies regions of H&E slides (i.e. an imaging biomarker) that contributes to its overall decision to classify a mouse as supersusceptible.

Model implementation
We modified our original model [45] because baseline model accuracy did not exceed pathologist performance. First, we decreased the second filter size from 3 £ 3 to 1 £ 1 (Layer #3 in Supplemental Table 2) to improve the model's capacity to learn relationships across feature maps of the previous layer while reducing the number of parameters. Second, we increased the initial filter size from 4 £ 4 to 7 £ 7 (Layer #1 in Supplemental Table 2) to eliminate the shifting caused by an even-dimensioned filter in the original model. This also increased the field of view for the initial set of filters. Further, this filter increased the perceptive field to a size that approximates the original two 3 £ 3 convolutions. Overall, the 1 £ 1 convolutions serve solely to learn and extract the relationship between feature maps while 7 £ 7 convolution exploits the spatial relationships in individual feature maps. Third, we increased the corresponding number of filter maps in the first layer from 36 to 48 (Layer #1 in Supplemental Each instance is individually subjected to feature extraction, resulting in instance features. Attention-based pooling computes attention weights (relative importance) of each of these instance vectors. c) Computed instance weights scale their respective instance features and are summed to a bag-level feature vector, which is classified as supersusceptible or not-supersusceptible. d) Attention weights are mapped back onto the tissue section image to bring attention to which areas of the tissue are being used for classification. e) An expert examines model-identified regions and identifies what the model attends to (i.e. "sees") leading to interpretable imaging biomarkers and potential knowledge discovery as well as validation of clinical practices. Table 2) to offset the increase in the receptive field size when changing the filter size from 4 £ 4 to 7 £ 7 and to increase the feature set extracted by the network. Finally, we added an additional convolutional layer with ten 3 £ 3 filters to refine and reduce the number of feature maps (Layer #5 in Supplemental Table 2). Overall, our architecture requires fewer parameters (389,340) than the original (1,231,574) without increasing training time (ours: 14598 §205s, original: 13216 §2250s, p=0.19 using a paired t-test) across six folds and exhibits higher performance. A technical summary of the baseline model and the proposed model are in Supplemental Table 2.
Models were optimized using Adam with b 1 = 0.9 and b 2 = 0.999, the learning rate of 0.0001, weight decay of 0.0005, and over 100 or 200 epochs. Due to class imbalances, our cross-validation approach was carried by randomly sampling 30 tissue sections from both supersusceptible (~10%) and not-supersusceptible (~20%) categories for the validation set in each fold and including the rest of the cases for the training of the model, known as Monte Carlo cross-validation [53]. To account for imbalanced training sets, the training procedure was modified to randomly select supersusceptible mice or not-supersusceptible mice with equal likelihood and then to randomly select a mouse from the selected category during every training iteration. Negative log-likelihood was used as a cost function in our implementation.

Model summary
A bag is created from a tissue section by sampling some number of image crops (Fig. 1b). Each instance is subject to feature extraction using the implementation described in Supplemental Table 2, resulting in instance embeddings (Fig. 1b). Attention-based pooling computes attention weights (i.e. relative importance) of each of these instance vectors (Fig. 1b). Computed instance weights scale their respective instance embeddings and summed to a bag-level embedding (Fig. 1c). This embedding is classified as supersusceptible or notsupersusceptible (Fig. 1c) with a cutoff of 0.5. Attention weights for each image crop can then mapped back onto the tissue section image to bring attention to which areas of the tissue are being used for classification (Fig. 1d). The model was implemented in Python using the PyTorch library.

Statistical methods
Model performance was evaluated using overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the ten-fold cross-validation described above. 95% confidence intervals for each statistic above were computed using bootstrapped samples of predictions (equal to the number of observations) with replacement (n = 1000). 97.5th and 2.5th percentiles were taken as bounds for confidence intervals. Pathologist and non-expert performance were similarly evaluated using accuracy, sensitivity, specificity, positive predictive value, and negative predictive value with confidence intervals. The proposed machine learning model was compared to the baseline using a paired t-test of the distributions of accuracy of each fold. Each procedure was carried out in MATLAB. Multivariate analyses used ANOVAs followed by Kruskal-Wallis post-tests in GraphPad Prism v8 for the comparison of the quantified imaging biomarker.

Lung granuloma patterns of M.tb-infected DO mice resemble humans
We observed many lung granuloma patterns in M.tb-infected DO mice (Fig. 2), mimicking the heterogeneity of granulomas in human pulmonary TB patients [54]. The lungs of supersusceptible DO mice, who 1) developed morbidity and mortality due to necropsy and 2) microbiologically-confirmed pulmonary TB within 8 weeks of infection, contained large coalescing regions of necrotic cellular, nuclear, and proteinaceous debris; neutrophils and macrophages; and tissue damage due to alveolar septal necrosis and capillary thrombosis (Fig. 2aÀf). Lymphocytes and plasma cells were present but not prominent. The lungs of DO mice who survived more than 8 weeks without morbidity/mortality (i.e. not-supersusceptible) showed different granuloma patterns. Granulomas were not necrotic and instead contained many perivascular and peribronchiolar lymphocytes, plasma cells, and foamy macrophages (Fig. 2gÀl) similar to most inbred mouse strains [55,56]. Interestingly, a small fraction (3À4%) of those DO mice developed histiocytic pneumonia, and lung cavities containing neutrophil and macrophage cellular debris surrounded by peripheral fibrosis (Fig. 2mÀo). Chronic, discrete, organizing granulomas were occasionally observed (Fig. 2pÀr). The lung patterns indicate that granuloma and/or lung tissue necrosis and neutrophilic influx are disease pathways of supersusceptibility, reflecting genetically controlled inflammatory responses to M.tb. These findings align with our prior work which showed that neutrophil chemokines (e.g. CXCL1, CXCL2, and CXCL5) and other innate inflammatory cytokines (e.g. Tumor Necrosis Factor) were significantly increased in the lungs of supersusceptible DO mice [23].

Model performance increases as the number instances included from each slide increases
Our first set of experiments explored how the baseline model would perform using different tile size and number of instances combinations (Supplemental Table 1). For each number of instances and tile size combination (Supplemental Table 1), a distinct dataset was generated. Each dataset consisted of a bag for each tissue section labeled with its associated diagnostic category (supersusceptible vs. not-supersusceptible). This resulted in 18 distinct datasets. Next, in order to determine optimal tile size and bag size configuration, a tenfold cross-validation of the baseline model [45] was performed using the same set of training parameters (Supplemental Fig. 2). Overall, these experiments show two trends. Comparing configurations in the same rows (in which bag size remains the same), both training losses and errors quickly converge, albeit more slowly for smaller tile sizes, validation error generally doesn't converge, and validation loss begins to rise after about the same epoch. This implies that tile size does not affect the performance of the model. Comparing configurations within the same column (where tile size remains the same), as the number of tiles increases, convergence is slower but overall validation loss and error diverge less. This implies that the number of tiles is an important factor for validation convergence. Intuitively, this makes sense, as 1) the filter sizes in the CNN are much smaller than any tile size, so the resulting feature maps change little with change in tile size, and 2) a greater number of tiles represents more spread across the slide.

Proposed machine learning model significantly outperforms baseline model in identifying supersusceptibility
Using these optimized parameters (32 £ 32 pixel tiles, mean/std bag size of 5000/1000), the proposed machine learning model (Supplemental Table 2) was subject to a ten-fold cross-validation and compared against the baseline model (Supplemental Fig. 3). The proposed machine learning model tends to perform much accurately (converging around 8% error) and converges at a faster rate compared to approximately the baseline model (approximately 16% error  . Lung lobes were fixed in formalin, paraffin-embedded, sectioned at 5mm, and stained with hematoxylin and eosin. Lungs from normal, non-infected mice are shown in panels a, b, c. Lungs from supersusceptible mice with granuloma necrosis with neutrophil influx and pyknotic nuclear debris are shown in panels d, e, f. Lungs from not-supersusceptible mice are a spectrum from mild (g, h, i) to moderate (j, k, l) lymphohistiocytic pneumonia to severe pneumonia with cavitation and fibrosis (m, n, o), and necrosuppurative granulomas (p, q, r). Images are magnified 15x, 40x, and 400x normal from left to right. We compared the model's performance to diagnose supersusceptibility using lung tissue sections against one expert board-certified veterinary pathologist (Gillian Beamer) with over ten years of experience studying lungs of mice experimentally infected with M.tb and one board-eligible third-year veterinary pathology resident (Cesar Piedra-Mora) with no experience in mouse TB model. The pathologists reviewed H&E stained lung sections blinded. The experienced pathologist (GB) directly evaluated all sections with no additional training, while the junior pathologist (CPM) studied 10 randomly selected images from each class (supersusceptible and not-supersusceptible) prior to evaluating all available images 1 week later.  (Fig. 3).

Automatically identifies pyknotic nuclei and nuclear debris as imaging biomarkers for supersusceptibility
Each slide was exhaustively divided into 32 £ 32 image crops, and their attention weights were computed using proposed machine learning model. A corresponding image heatmap was generated in which image crops were replaced by their attention weights. Resulting examples can be seen in Figs. 4 and 5 and Supplemental Fig. 4. In order to ascertain interpretability, potential as automatically identified imaging biomarkers, and clinical validity, the expert pathologist (GB) examined resulting heatmaps alongside respective tissue section counterparts and identified what the model was attending to (Fig. 1e). She concluded that the primary feature being identified by the proposed machine learning model was pyknotic nuclei and nuclear debris (Fig. 5). One other board-certified veterinary pathologist (Famke Aeffner) corroborated these observations by judging the presence of necrosis from 30 images sampled model-identified regions (like Fig. 5aÀd) and non-model-identified regions (like Fig. 5eÀh). They agreed with the model's identification of necrosis 73.33% of the time.

Non-experts identify supersusceptible mice using imaging biomarkers with high accuracy
After generating heatmaps for each digital tissue section, two non-experts (Thomas Tavolara and Khalid Niazi) were trained on a subset of slides (

Automatic quantification of imaging biomarkers provides continuous data for statistical analyses
Pathologists cannot quantify cellular, nuclear, or stromal features within a tissue section with accuracy or reproducibility [57,58]. Generally, they attempt to judge extent of disease by relying on grades or scales that are estimates [59,60]. The automatically identified lung imaging biomarker was quantified (Supplementary Methods) for tissue sections in each mouse as a ratio of necrotic pyknotic affected regions to non-necrotic tissue (Fig. 6). The imaging biomarker was significantly increased in the lung sections of M.tbinfected supersusceptible DO mice (n = 148) than not-supersusceptible (n = 266) M.tb-infected DO mice, M.tb-infected C57BL/6J inbred mice (n = 10), and non-infected DO mice.

Discussion
Each DO mouse carries genome segments from 8 inbred founder strains, resulting in a mosaic across the chromosomes (Supplementary Fig. 1). The genetic diversity of DO mice is equivalent to humans [8], and this genetic diversity contributes to unique and wide phenotypes [20] that are not observed within or between common inbred strains [5,6,19,22À30]. Even our work with panels of founder and Collaborative Cross inbred strains [29] do not produce the response range in DO mice [23] and our unpublished data. We showed that supersusceptible DO mice develop pulmonary TB with large necrotizing granulomas and abundant M.tb bacilli, while those that survive longer develop non-necrotic lymphocyte-rich granulomas. Published elsewhere, we confirmed that DO mice are not immune deficient or highly susceptible to other mycobacteria, and they generate antigenspecific immunity against M.tb [16,23,24]. Together, these results support our conclusion that DO mice are a valuable animal model of TB. Since we can now rigorously quantify necrotic and pyknotic debris within lung granulomas using the method here, our ongoing and future work can focus identifying novel mechanisms of M.tbinduced granuloma necrosis. This is a great translational benefit of DO mice and of computational pathology because tissue and cellular features of histology images cannot be accurately quantified by pathologists in large image data sets.
Although deep learning in the context of computer vision has achieved many great feats, from human-level object detection to realistic synthetic image generation and self-driving vehicles, it has still generally been eluded by knowledge discovery, in which other fields have seen some successes [61,62]. The results of the proposed machine learning model suggest a path for such knowledge discovery in medical image analysis. Here, we did not use any manual annotations to guide the MIL. Instead, two outcomes (classes) were used solely À supersusceptible, defined as rapid pulmonary TB within 8 weeks of infection; and not-supersusceptible, defined as no clinical signs for at least the same duration. This identified an imaging biomarker in granulomas that 1) Automatically diagnoses supersusceptibility; 2) Serves as a tool (heatmap) for non-experts to diagnose supersusceptibility; 3) Was clinical interpretable by two board-certified veterinary pathologists as pyknotic debris; and 4) Quantifies pyknotic debris for statistical comparisons. Our results imply that computational approaches can produce and use the same imaging biomarkers that pathologists use, mimicking pattern recognition and decision-making of pathologists reading histology slides. Thus, our work sheds light on the "black box" of computational pathology and instills confidence that artificial intelligence can make accurate diagnoses using interpretable, biologically relevant histopathology imaging biomarkers.
We recognize three major shortcomings of conventional deep learning approaches [34,36À41,58,63,64] which the proposed MIL model overcomes. First, conventional deep learning models require strong labels [34]. Labels are classes or identities we assign to images. Strong labels can be thought of as any label that can be delineated in an image (a.k.a. image annotations). For example, in the context of medical image analysis, a strong label could be an outline or segmentation of an organ or tissue structure in MRI or CT, or it could be anatomical structures or cell types in histology. The proposed MIL model directly addresses this shortcoming of conventional approaches by requiring weak labels, labels assigned to whole image rather than its parts. Examples of weak labels include a diagnosis, status, tumor malignancy/benignness, cancer stage, or degree of neurodegeneration. In the context of the problem addressed here, the weak labels of supersusceptible and not-supersusceptible are utilized not only because they are available to us but also because tissue-level knowledge (i.e. strong labels) is not readily available À there are no tissue Fig. 5. Primarily nuclear debris is being identified. Heatmaps with their corresponding tissue region to the right (H&E) from eight supersusceptible mice. Brighter (middle column) or redder (right column) regions correspond to areas of the images that correspond to supersusceptible features (i.e. image biomarkers). Note that primarily nuclear debris is being identified and that healthy tissue is not being identified.

Table 2
Agreement between two non-experts using imaging biomarkers.  areasorcelltypeswhichwecaneasilyandconfidentlyannotateassupersusceptible or not-supersusceptible. The distinction is key for the automated analysis of medical images, as acquisition of relevant annotations (i.e.domainknowledge)istimeconsumingandthusoneofthebarriersfor artificialintelligenceinmedicine. Second, conventional deep learning models fall short, as they do not efficiently process digital tissue slides, which are in the order of gigapixels. TB detection studies that utilize chest x-ray images often rely on CNNs for fibrosis detection [36À41]. Chest x-ray image sizes are in the order of thousands by thousands of pixels (i.e. 1000 £ 1000 to 5000 £ 5000), and as is typical in CNN applications, are resized to small images (typically 200 £ 200 to 300 £ 300) 1) for the purpose of fitting the fixed-input size for their CNN model and 2) because of lack of memory on current GPUs. Though not explicitly justified in the literature, the underlying assumption in resizing images is that smaller details (which are lost from resizing) aren't necessary for CNN models. Unlike x-rays, digital pathology slides are in the order of billions of pixels. To resize an entire slide to the fixed-input size of a deep learning model would result in loss of nearly all cell and anatomical structure information. Thus, we cannot simply resize the input image. Typical solutions in digital pathology include extracting high-power fields (akin to fields of view a pathologist might analyze under the microscope) or extracting tiles (small images cropped from the slide). In the former, high power fields are annotated by a pathologist, thereby returning to the problems of strong labeling discussed earlier. In the latter, individual tiles are assigned labels based on their anatomical content. However, processing hundreds of thousands of image tiles per slide is inefficient in addition to requiring strong labeling. Moreover, it is not explicitly clear how model-predicted cell and tissue level labels relate to weaker clinical labels nor how multiple tile-level predictions should be aggregated into clinical labels. The proposed MIL model resolves the magnitude of the problem (i.e. the large image size) by sampling digitized tissue sections (without resizing) for small 32 £ 32 image tiles. It also resolves the question of how to aggregate said tiles through an automatically learned aggregation mechanism À namely, attention pooling.
Third, conventional deep learning models are generally not interpretable. Lack of interpretability is an oft criticized aspect of CNNbased deep learning in image analysis [58,63,64]. Though CNN models have been shown to achieve near human-level performance on several computer vision tasks in specific domains, they generally lack the ability to explain how they decide. Thus, such approaches are often dubbed 'black-boxes.' This black-box nature is particularly troubling in biomedical research and medicine, as scientists and clinicians need to know how and why a model makes decisions in order to understand some underlying phenomenon (i.e. knowledge discovery) and to make informed clinical decisions. Given this crucial shortcoming of CNN-based deep learning, there exist several tools to give some insight into what they "see." Class activation mapping (CAM) [42] was one of the first of these tools. This method allowed CNN models to highlight which parts of an input image contribute to its overall classification. Grad-CAM [43] immediately improved on CAM by allowing for more complex CNN models. There was further improvement in a method dubbed Grad-CAM++ [44]. Several studies examining CNNs as potential clinical tools for TB utilized these methods to visualize which parts of chest X-rays were being utilized for their model's decision [36À39]. However, in digital pathology, these methods cannot be applied, as they require entire images as input to highlight decision-supporting regions. Digital slides are too large to be processed in such a manner and cannot be resized for reasons discussed. Further, these methods focus largely on strong labels in multi-class problems and thus cannot apply to the current problem. The proposed MIL model resolves these issues of interpretability through the automatically learned attention pooling mechanism [45]. Automatically identified instances give insight into what the proposed machine learning model is "seeing." In the context of medical image analysis, this attention pooling provides a useful mechanism that does not require manual annotation of microscopic features. Here, it automatically discovered an imaging feature ("biomarker") diagnostic for supersusceptibility À namely, pyknotic nuclei and nuclear debris. Despite the proposed machine learning model not being aware, granuloma necrosis is used by pathologists to diagnose pulmonary TB in human patients and is a desirable feature of experimental TB in animal models. Thus, our automatic method mimics how pathologists examine biopsy, autopsy, and experimental tissues to make diagnostic decisions, providing validation and clinical relevance. Importantly, the proposed machine learning model did not identify healthy lung tissue. This affirms the MIL assumption that positive bags may contain instances similar to those found in negative bags. In this problem, those instances are healthy lung tissue (Fig. 5eÀh). Healthy tissue is present in both supersusceptible and not-supersusceptible M.tb-infected DO mice. It therefore should not be a discriminatory feature identified by the proposed machine learning model. And indeed, this is the case. It is obvious from the heatmap that areas of healthy lung tissue and never attended to (Figs. 4 and 5eÀh).
As a result of automatically identifying imaging biomarkers, the proposed machine learning model was also able to generate heatmaps corresponding to presence and intensity of features indicative of supersusceptibility. This resulted in simplified images which nonexperts used to classify supersusceptibility or not with greater accuracy than an expert using H&E. This is understandable, as pathologists are trained for years to differentiate among thousands of different tissues, cell types, and other features in more than one type of stain. Comparatively, the proposed machine learning model learns only to focus a few of these features, primarily pyknotic nuclei and nuclear debris. Additionally, it samples across the entire slide rather than a small, localized subset of regions (like pathologists). Nonexperts benefit from heatmaps by being able to focus on a specific feature while being undistracted by irrelevant features. Both nonexpert readers identified two key features of heatmaps in their training set À density and clustering. In general, supersusceptible heatmaps were identifiable by multiple large, dense clusters. Heatmaps from the not-supersusceptible class usually were either not very dense or did not form clusters ( Fig. 4; Supplemental Fig. 4).
The proposed machine learning model has limitations. For example, it cannot identify imaging biomarkers of non-supersusceptibility, likely because the non-supersusceptible class is homogenous, containing few morphological subtypes and also non-infected DO mice. Future work will yield imaging biomarkers of intermediate and longterm resistance to M.tb. Currently, we are increasing our data set and investigating how to automatically discover important features of other classes of M.tb-infected DO mice, such susceptible and resistant DO mice (Table 1).
Overall, our results show that computational approaches can automatically identify and utilize the same imaging biomarkers that pathologists use, mimicking expert human pattern recognition and decision-making. Our work sheds light on the "black box" of computational pathology and instills confidence that artificial intelligence methods can make accurate diagnoses using interpretable histopathology features. In future studies, we intend to develop a multi-class version of this framework to identify and quantify unique granuloma features of highly resistant DO mice, and of vaccinated DO mice. Finally, we will explore how the proposed machine learning model performs in related scenarios and whether model-identified imaging biomarkers are interpretable quantifiable.

Contributors
TET contributed to data curation, formal analysis, methodology, software, validation, writing the original draft, and editing the manuscript. MKKN contributed to methodology, formal analysis, and review and editing of the manuscript. MG contributed to data collection. CPM contributed to review and editing of the manuscript. DG contributed to data analysis and review and editing of the manuscript. GB contributed to data collection, data curation, formal analysis, conceptualization, funding acquisition, project administration, and review and editing of the manuscript. MNG contributed to conceptualization, formal analysis, funding acquisition, methodology, project administration, and review and editing of the manuscript.

Data sharing statement
All data and code utilized for development and validation is available at github.com/cialab/imaging_biomarker to anyone who wishes to utilize it for any purpose following publication. All raw imaging data will be made available upon request by contacting mniazi@wakehealth.edu immediately after publication.

Declaration of Competing Interest
Mr. Tavolara has nothing to disclose. Dr. Niazi has nothing to disclose. Ms. Ginese has nothing to disclose. Dr. Piedra-Mora has nothing to disclose. Dr. Gatti has nothing to disclose. Dr. Beamer has nothing to disclose. Dr. Gurcan has nothing to disclose.