{'fraction_non_alphanumeric': 0.03165317733792772, 'fraction_numerical': 0.01394342495392259, 'mean_word_length': 5.357615894039735, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adjusting for Confounding in Unsupervised Latent Representations of Images

Biological imaging data are often partially confounded or contain unwanted variability. Examples of such phenomena include variable lighting across microscopy image captures, stain intensity variation in histological slides, and batch effects for high throughput drug screening assays. Therefore, to develop"fair"models which generalise well to unseen examples, it is crucial to learn data representations that are insensitive to nuisance factors of variation. In this paper, we present a strategy based on adversarial training, capable of learning unsupervised representations invariant to confounders. As an empirical validation of our method, we use deep convolutional autoencoders to learn unbiased cellular representations from microscopy imaging.


Introduction
Recently, there has been growing evidence showing that machine learning systems can lead to poor generalisation due to inductive biases [1]. For instance, it has been shown that algorithms trained on biased data sets end up discriminating against minorities when allocating resources and/or opportunities -see e.g. [2,3]. This has led to the realisation that algorithmic decision making needs to go beyond "simple" minimisation of errors, in order to learn "fair" hypotheses.
For biomedical applications, solving cognitive tasks by learning supervised deep representations requires large amounts of labelled data. However, ground truth annotations are extremely expensive and time consuming since they require domain experts. Furthermore, the collection of high confidence human labelling is hindered by both interobserver and intraobserver variability -see e.g. [4,5,6,7].
Given such considerations, in this work we develop a deep learning method capable of learning unsupervised representations of images that encode meaningful biological knowledge whilst being invariant to specified confounding variables. We enforce fairness into latent representations by exploiting adversarial training to remove "noise signals" induced by confounders.
To validate our approach, we consider a high throughout microscopy imaging assay that captures morphological perturbations in cancer cell lines exposed to a compendium of drugs. Microscopy assays enable the analysis of cellular phenotypes through quantification of cell morphology. In particular, profiling of small molecules has the potential to deepen our understanding into mechanisms of action for specific compounds, thereby enabling faster drug development cycles [8].

Problem Statement
We model our learning task as follows. We are given a data set of images x i ∈ IR h×w×c where (h, w, c) denote height, width, and number of colour channels, respectively. Images are sampled from a data generating distribution characterised by the presence of both informative M and nuisance factors S, with some M 's confounded by S (Figure 1). Similar to previous work using neural networks to learn fair classifiers on toy and text data [9,10,11], we aim to learn, for each image x with associated nuisance factor s, a latent representation z ∈ IR d (where d h · w · c), independent of s. First, we train a convolutional autoencoder (CAE) to learn the reconstructionx = f (x, θ). The CAE is comprised of a cascade of two modules, where the encoder is estimating the posterior p(z | x) and the decoder is reconstructing the input image. Furthermore, we use an adversarial network g(z, w) to estimate p(s | z). Given the CAE and adversarial losses (L CAE , L adv ), we introduce the joint objective E: Therefore, deep representations which are invariant to s can be learned by solving the following min/max optimisation problem [9,10,11]: The architecture used for adversarial learning is visualised in Figure S1. All implementation details about our training protocol can be found on the GitHub repository accompanying this work (https: //github.com/Nellaker-group/FairUnsupervisedRepresentations).

Data
We used the BBBC021v1 image set [12], available from the Broad Bioimage Benchmark Collection [13]. These data are comprised of microscopy images capturing phenotype changes in breast cancer cell lines, after being exposed to several drugs at different concentrations. Phenotypes were captured by labelling cells with DAPI, Tubulin, and Actin, thereby generating a triplet of single channel images (one per fluorescent marker) for each treatment. Within such a resource, imaged across 10 weeks (batches), a subset of treatments have been annotated with a corresponding molecular mechanism of action (MOA). We focus our attention to 92 treatments obtained after removing the two mechanisms of action (cholesterol lowering and kinase inhibitors) that appear in single batches -see [12,13,14] for details.
The BBBC021v1 image set was specifically annotated as benchmark data to validate image based profiling methods and, consequently, this data set has been extensively analysed. For instance, recent works have developed deep learning approaches heavily reliant on transfer learning, in order to accurately predict MOA labels -see, e.g. [15,16,17].
For each image in the BBBC021v1 set, we detected cell nuclei using the algorithm difference of Gaussians [18] on the DAPI channel. We cropped patches of 128 × 128 pixels centred around each nucleus, and annotated 128 × 128 × 3 images by concatenating patches from the DAPI, Tubulin, and Actin channels. Representative images used to develop our model are visualised in Figure S2.

Results
Contrary to previous methods, this work is concerned with learning unsupervised representations of cells, by means of deep convolutional autoencoders. The CAE was trained with Adam [19] for 200 epochs, achieving a training (test) loss of 0.00297 (0.00297). Furthermore, as qualitative assessment, in Figure S3 we display some (test) reconstructions. Our CAE has not only learned about the global structure of images, but it is also capable of capturing fluorescence variance across cells, as well as intracellular luminescence gradients.
In order to quantitatively assess whether CAE representations are capturing meaningful biological knowledge, we implemented an approach similar to previous works [15,16]. We used CAE codings as feature vectors in a three nearest neighbour (3NN) classifier to predict MOA labels for the subset of images with annotations of molecular mechanisms (92 treatments). We assessed the expressiveness of our CAE representations by implementing a leave one out cross validation protocol, and compared the results against null accuracy. As can be seen in Table 1, learned CAE representations capture meaningful biological knowledge as well as experimental confounding, since we were able to accurately predict MOA labels and discriminate treatments according to their batch (week). The presence of a strong batch effect has been documented in these data before, but no attempt, to the best of our knowledge, has been undertaken to precisely quantify the extent to which such nuisance factor is boosting MOA accuracies, or whether confounding knowledge can be removed from learned representations [16]. The existence of a strong confounder is also manifest in Figure S4, where we display tSNE projections [20] of CAE codings for the (92) treatments with annotated mechanism of action. Figure S4 shows several sets of data points which are clustered according to their MOA label as well as their imaging batch (also see [16], Figures S1 and S2).
In this work we show that by means of adversarial training it is possible to remove confounding encoded in unsupervised cellular representations, whilst preserving biological expressiveness. Our results also represent empirical estimates that quantify to which extent the batch effect confounding is helping the MOA classification task. We used a training protocol similar to previous work [9,11], by implementing a min/max optimisation that minimises the reconstruction loss whilst removing batch effects from CAE representations (Section 2). This protocol is characterised by the presence of the hyperparameter λ, where λ > 1 emphasises fairer, less biased representations, whilst resulting in higher CAE reconstruction losses. To assess the quality of our latent representations after adversarial training, we, once more, used CAE codings as feature vectors for a nearest neighbour classifier to predict MOA labels and imaging batches. We analysed how MOA and batch accuracies change across different values of λ, and the results of our experimentation are collected in Table 1. The adversarial programme was successful in removing the batch effect confounder, whilst preserving meaningful biological knowledge helpful for the MOA classification task. Higher values of λ result in reductions of the CAE codings' power to discriminate batches. At λ = 50, our CAE codings are uninformative for the batch effect confounder, whilst still capable of successfully inferring MOA labels about 4× better than random chance. Furthermore, the decrease in MOA accuracies represent additional evidence for the strength of the confounder present in the BBBC021v1 image set.

Discussion
We presented an approach capable of learning unbiased representations that are biologically meaningful and invariant to uninformative factors of variation. Data representations learned by supervised approaches are intrinsically constrained to the set of cellular phenotypes annotated for training. Our method, based on the unsupervised learning paradigm, does not suffer from such a drawback, and also supports the identification of novel (i.e. currently unknown) phenotypes.
Contrary to previous adversarial approaches focusing on supervised learning [9,10,11], we demonstrated the ability to learn unsupervised representations void of nuisance knowledge whilst retaining as much biological information as possible. This is fundamental across all domains of machine learning where, in order to guarantee good generalisation and unbiased predictions, it is crucial to learn data representations insensitive to known confounders. In particular, this is of paramount importance for machine learning deployment in health care. Identifying and mitigating biases is crucial in order to avoid developing models that mirror human prejudice and/or learn confounds present in data [21].
We have shown it is possible to remove nuisance variation encoded in a categorical variable (assuming 10 distinct values) using an adversarial approach. However, this method can be trivially generalised to continuous factors by using an adversarial regressor ( Figure S1). We are currently analysing an additional imaging data set, derived from the Genotype Tissue Expression (GTEx) Consortium [22]. This resource is comprised of histological slides capturing a range of phenotypes across 53 human tissues, in approximately 800 individuals. Using our approach to learn CAE codings, we have discovered the presence of a continuous confounder (ischemic time), which is correlated with several dimensions of our unsupervised representations (Pearson's r = 0.18, p = 4.9 × 10 −150 ). We are currently implementing our adversarial learning protocol to remove the ischemic time confounder.

Code and Data Availability
The BBBC021v1 image set used in this work is freely available from the Broad bioimage benchmark collection web server (https://data.broadinstitute.org/bbbc/). Furthermore, this paper comes with a dedicated GitHub repository (https://github.com/Nellaker-group/ FairUnsupervisedRepresentations) where we have deposited all the scripts used for the analyses, and the weights of our CAEs. This work was implemented using Python 3 and the deep learning framework PyTorch [23]. Figure S2: Examples of data instances used to develop our approach. We trained our CAE using 128 × 128 × 3 images, centred on nuclei, obtained by stacking patches from the DAPI (R), Tubulin (G), and Actin (B) channels. The "R channel" is activated by cell nuclei whilst the "G channel" highlights the characteristic Tubulin halo surrounding each nucleus. Week (Batch)

CAE Reconstructions
Week 4 Week 8 Week 10 Week 9 Week 1 Week 7 Week 2 Week 6 Week 3 Week 5 Figure S4: 2D projections of CAE's latent representations, before adversarial training. Before implementing an adversarial optimisation programme, several sets of data points (representing treatments) cluster according to their MOA label (knowledge) as well as imaging batch (confounder).