{'fraction_non_alphanumeric': 0.03611222008492723, 'fraction_numerical': 0.010924400246797082, 'mean_word_length': 4.803285593934288, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Through a fair looking-glass: mitigating bias in image datasets

With the recent growth in computer vision applications, the question of how fair and unbiased they are has yet to be explored. There is abundant evidence that the bias present in training data is reflected in the models, or even amplified. Many previous methods for image dataset de-biasing, including models based on augmenting datasets, are computationally expensive to implement. In this study, we present a fast and effective model to de-bias an image dataset through reconstruction and minimizing the statistical dependence between intended variables. Our architecture includes a U-net to reconstruct images, combined with a pre-trained classifier which penalizes the statistical dependence between target attribute and the protected attribute. We evaluate our proposed model on CelebA dataset, compare the results with a state-of-the-art de-biasing method, and show that the model achieves a promising fairness-accuracy combination.


Introduction
Due to their increased usage within myriad software applications, artificial intelligence algorithms now influence many aspects of people's lives, particularly when they are embedded into decision-support tools used by educators, government agencies, and various industry sectors. Thus, it is crucial to make sure that these algorithms are scrutinized to ensure fairness and remove unjust biases. Bias has been shown to exist in several deployed AI systems, including the well known Correlational Offender Management Profiling for Alternative Sanctions (COMPAS). COMPAS is an automated decision making system used by the US criminal justice system for assessing a criminal defendant's likelihood of re-offending. By exploring the risk scores assigned to individuals, this system has been shown to be biased against African Americans [4]. Other examples include a version of Google's targeted advertising system in which highly paid jobs were advertised more frequently to men vs. women [12].
Bias in computer vision is a major problem, often stemming from the training datasets used for computer vision models [24]. There is evidence suggesting the existence of multiple types of bias, including capture and selection bias, in popular image datasets [25]. The problems arising from bias in computer vision can manifest in different ways. For instance, it is observed that in activity recognition models, when the datasets contain gender bias, the bias is further amplified by the models trained on those datasets [36]. Face recognition models may exhibit lower accuracy for some classes of race or gender [2].
Works such as [26,33] suggest methods to mitigate bias in visual datasets. Several studies have deployed GANs for bias mitigation in image datasets. For example, [22] modified the value function of GAN to generate fair image datasets. FairFaceGAN [10] implements a facial image-to-image translation, preventing unwanted translation in protected attributes. Ramaswamy et al. propose a model to produce training data that is balanced for each protected attribute, by perturbing the latent vector of a GAN [19]. Other studies employing GANs for fair data generation include [3,23].
A variety of techniques beyond GANs have been applied to the problems of fairness in AI. A deep information maximization adaptation network was used to reduce racial bias in face image datasets [28], and reinforcement learning was used to learn a race-balanced network in [27]. Wang et al. propose a generative few-shot cross-domain adaptation algorithm to perform fair cross-domain adaption and improve performance on minority category [30]. The work in [32] proposes adding a penalty term into the softmax loss function to mitigate bias and improve fairness performance in face recognition. Quadriento et al. [17] propose a method to discover fair representations of data with the same semantic meaning of the input data. Adversarial learning has also successfully been deployed for this task [29,34].
This paper addresses the issue of a decision-making process being dependent on protected attributes, where this dependence should ideally be avoided. From a legal perspective, a protected attribute is an attribute upon which discrimination is illegal [16], e.g. gender or race. Let D = (X , S, Y) be a dataset, where X represents unprotected attributes, S is the protected attribute, and Y be the target attribute. If in the dataset D, the target attribute is not independent of the protected attribute (Y ⊥ S), then it is very likely that the decisionsŶ made by a decision-making system which is trained on D, is also not independent of the protected attribute (Ŷ ⊥ S).
We propose a model to reconstruct an image dataset to reduce statistical dependency between a protected attribute and target attribute. We modify a U-net [20] to reconstruct the image dataset and apply the Hilbert-Schmidt norm of the cross-covariance operator [6] between reproducing kernel Hilbert spaces of the target attribute and the protected attribute, as a measure of statistical dependence. Unlike many previous algorithms, our proposed method doesn't require training new classifiers on the unbiased data, but instead reconstructing images in a way that reduces the bias entailed by using the same classifiers.
In Section 2 we present the problem, the notion of independence, and our proposed methodology. In Section 3 we describe the CelebA dataset and the choice of feature categorization, introduce the baseline model with which we compare our results [19], our model's implementation details, and finally present the experiments and results.
Bias mitigation methods can be divided into three general categories of pre-process, inprocess, and post-process. Pre-process methods include modifying the training dataset before feeding it to the machine learning model. In-process methods include adding regularizing terms to penalize some representation of bias during the training process. Finally, post-process methods include modifying the final decisions of the classifiers [8]. Kamiran and Calders [11] propose methods such as suppression which includes removing attributes highly correlated with the protected attribute, reweighing, i.e. assigning weights to different instances in the data, and massaging the data to change labels of some objects. Bias mitigation methods often come at the expense of losing some accuracy, and these preliminary methods usually entail higher fairness-utility cost. More sophisticated methods with better results include using generative models to augment the biased training dataset with unbiased data [19], or training the models on entirely synthetic unbiased data [18]. Wang et al. [31] provide a set of analyses and a benchmark to evaluate and compare bias mitigation techniques in visual recognition models.

Methodology
Consider a dataset D = (X , S, Y), where X is the set of images, Y = {+1, −1} is the target attribute such as attractiveness, and S = {A, B,C, ...} is the protected attribute such as gender. Assume there exists a classifier f : (X ) → Y, such that the classifier's prediction for target attribute is not independent from the protected attribute, i.e. f (X ) ⊥ S. Our objective is to design a transformation g : X → X , such that 1) f ( X ) ⊥ S, i.e. the classifier's predictions for target attribute is independent of the protected attribute , and 2) f ( X ) ≈ f (X ), i.e. the classifier still achieves high accuracy.
In other words we want to train a network to transform our original images, such that if the classifiers that are trained on the original and unmodified images, are used to predict the target attribute (attractiveness in our example) from the transformed version of an image, they still achieve high accuracy, while the predictions of those classifiers are independent of the protected attribute (gender in our example). It should be noted that we are not seeking to train new classifiers, but rather only aim to modify the input images. This is a main distinction between our methodology and most of other techniques (e.g. [17] and [19]), in which the process includes training new classifiers on modified new image datasets and achieving fair classifiers.
Our proposed model consists of a U-net [20] as the neural network that transforms the original images. This type of network was originally proposed for medical image segmentation, and has been widely used since its introduction. The encoder-decoder network consists of two paths, a contracting path consisting of convolution and max pooling layers, and a consecutive expansive path consisting of upsampling of the feature map and convolutions. Contrary to [20] where each image is provided with a segmented image label, we provide our U-net with the exact same image as the label, and alter the loss function from cross-entropy to mean squared error, so that the network gets trained to produce an image as close to the original image as possible, in a pixel-wise manner.
While some previous fairness studies consider decorrelating the target attribute from the protected attributes, what must be ultimately sought however, is independence between the protected attribute and the target attribute. Dealing with two random variables which are uncorrelated is easier than independence, as two random variables might have a zero correlation, and still be dependent (e.g. two random variables A and B with recordings A = [−2, −1, 0, 1, 2] and B = [4, 1, 0, 1, 4] have zero covariance, but are apparently not independent). Given a Borel probability distribution P ab defined on a domain A × B, and respective marginal distributions P a and P b on A and B, independence of a and b (a |= b) is equal to P xy factorizing as P x and P y . Furthermore, two random variables a and b are independent, if and only if any bounded continuous function of the two random variables are uncorrelated [7].
Let F and G denote all real-value functions defined on domains A and B respectively. In their paper Gretton et al. [6] define the Hilbert-Schmidt norm of the cross-covariance operator: HSIC(P ab , F, G) := ||C ab || 2

HS
(1) where C ab is the cross-covariance operator. They show that if ||C ab || 2 HS is zero, then cov( f , g) will be zero for any f ∈ F and g ∈ G, and therefore the random variables a and b will be independent. Furthermore, they show if Z := (a 1 , b 1 ), ..., (a n , b n ) ∈ A × B are a series of n independent observations drawn from P ab , then a (biased) estimator of HSIC is [6]: where H, K, L ∈ R n×n , K and L are Gram matrices [9], K i j := k(a i , a j ), L i j := l(b i , b j ), k and l are universal kernels, and H i j := δ i j − n −1 centers the observations in feature space. We use Hilbert-Schmidt independence criteria to penalize the model for dependence between the target attribute and the protected attribute.

Training Loss Function
We seek to modify a set of images, such that 1) the produced images are close to the original images, and 2) the predicted target attribute is independent from the predicted protected attribute. In the optimization problem, image quality (1) is measured by pixel-wise MSE loss. For independence (2), consider our U-net network as a mapping from original image to the transformed image, i.e. U w (x) = x. Consider also a function h : . Given X representing a batch of N training images and X representing the transformed batch, our formal optimization problem is as follows: where N is the number of samples, C is the number of channels of an image, W is the width of an image, H is the height of an image, and λ is the parameter that controls the trade-off between accuracy of the transformed images and independence (fairness). In practice, the mapping function U w that we use is a U-net, the function h(·) is a pre-trained classifier with two outputs h 1 and h 2 , each being the output of a Sigmoid function within the range of [0, 1], where h 1 = P(Y = 1|X) (a vector of size N), and h 2 = P(S = 1|X) (also a vector of size N), and HSIC(·, ·) denotes Hilbert-Schmidt Independence Criteria. Figure 1 shows the network architecture and a schematic of the training procedure. Consider a batch of original images X entering the U-net. The U-net then produces the reconstructed images U w (X) = X. To calculate the image accuracy part of the loss function, the original image batch X is provided as label and the Mean Squared Error is calculated to measure the accuracy of the reconstructed images. The ResNet component in Figure 1 is our h(·) function as described before, which is a pre-trained ResNet classifier that takes as input a batch of images and returns two probability vectors. The second part of the loss function, independence, is calculated by entering the reconstructed images X into this ResNet classifier, and calculating the HSIC between the two vectors.
As noted before, the image dataset is reconstructed in a way that using them on the original biased classifiers, will result in an improvement in classifications. This is dissimilar to some previous works such as [19] and [17], in which the model training process includes augmenting the original dataset with generated images and training new fair classifiers [19], or discovering fair representations of images and subsequently training new classifiers [17].

Experiments
In this section, we test the methodology described in Section 2 on CelebA dataset [13]. We first introduce the CelebA dataset and the attribute categories in CelebA. We then describe the implementation details of our model. Subsequently, the method described in Ramaswamy et al. [19] and the two versions of it that we use as baseline models to compare our results with are introduced. Finally, we introduce evaluation metrics and present the results.

CelebA dataset
CelebA is a popular dataset that is widely used for training and testing models for face detection, particularly recognising facial attributes. It consists of 202,599 face images of celebrities, with 10,177 identities. Each image is annotated with 40 different binary attributes describing the image, including attributes such as Black_Hair, Pale_Skin, Wavy_Hair, Oval_Face, Pointy_Nose, and other attributes such as Male, Attractive, Smiling, etc. The CelebA dataset is reported to be biased [35]. In this experiment, we consider Male attribute as the protected attribute (with Male = 0 showing the image does not belong to a man and Male = 1 showing the image belongs to a man), and Attractive to be the target attribute. We divide the dataset into train and test sets, with train set containing 182,599 and test set containing 20,000 images. In the training set, 67.91% of images with Male = 0 are annotated to be attractive (Attractive = 1), while only 27.93% of images with Male = 1 are annotated as being attractive (Attractive = 1). This shows bias exists against images with Male = 1.
In order to compare our results with [19], we follow their categorization of CelebA attributes. Leaving out gender (Male) as the protected attribute, among the rest 39 attributes in CelebA dataset, [19] eliminates some attributes such as Blurry and Bald as they contain less than 5% positive images. The remaining 26 attributes is subsequently categorized into three groups. inconsistently-labeled attributes are the ones that by visually examining sets of examples, the authors often disagree with the labeling and could not distinguish between positive and negative examples [19]. This group includes attributes such as Straight_Hair, and Big_Hair. The second group of attributes are the ones that are called gender-dependent and the images are labeled to have (or not have) attributes based on the perceived gender [19]. These include attributes such as Young, Arched_Eyebrows and Receding_Hairline. Finally, the last group of attributes are called gender-independent. These attributes are fairly consistently labeled and are not much dependent on gender expression. This group includes attributes such as Black_Hair, Bangs, and Wearing_Hat. The list of all attributes is provided in supplementary material.
In order to compare our results with [19], we follow their categorization of CelebA attributes. Leaving out gender (Male) as the protected attribute, among the rest 39 attributes in CelebA dataset, [19] eliminates some attributes such as Blurry and Bald as they contain less than 5% positive images. The remaining 26 attributes is subsequently categorized into three groups. inconsistently-labeled attributes are the ones that by visually examining sets of examples, the authors often disagree with the labeling and could not distinguish between positive and negative examples [19]. This group includes Straight_Hair, Big_Lips, Big_Nose, Oval_Face, Pale_Skin, and Wavy_Hair. The second group of attributes are the ones that are called gender-dependent and the images are labeled to have (or not have) attributes based on the perceived gender [19]. These include Young, Arched_Eyebrows, Attractive, Bushy_Eyebrows, Pointy_Nose, and Recedin Finally, the last group of attributes are called gender-independent. These attributes are fairly consistently labeled and are not much dependent on gender expression. This group of attributes include Black_Hair, Bangs, Blond_Hair, Brown_Hair, Chubby, Wearing_Earrings, Bags_Under_Eyes, Eyeglasses, Gray_Hair, High_Ch Mouth_Slightly_Open, Narrow_Eyes, Smiling, and Wearing_Hat.

Attribute classifiers
For attribute classifiers, we use ResNet-18 pre-trained on ImageNet, in which the last layer is replaced with a layer of size one, along with a Sigmoid activation for binary classification. We train all models for 5 epochs with batch sizes of 128. We use the Stochastic Gradient Descent optimizer with a learning rate of 1e-3 and momentum of 0.9. We use a step learning rate decay with step size of 7 and factor of 0.1. After training, we will have 26 classifiers that receive an image and perform a binary classification on their respective attribute.

Implementation details
As shown in Figure 1, a ResNet-18 network is used to accompany the U-net to produce predictions for Male and Attractive. Prior to training the U-net, the ResNet-18 [21] which is pre-trained on ImageNet, is modified by replacing its output layer with a layer of size two, outputing the probability of attractiveness and gender. The ResNet-18 is then trained for 5 epochs on the train set, with a batch size of 128. We use the Stochastic Gradient Descent optimizer with a learning rate of 1e-3 and momentum of 0.9. We use a step learning rate decay with step size of 7 and factor of 0.1. After the ResNet is trained and prepared, we train the U-net as described in Section 2 on the train set. The detailed architecture of the U-net is described in Supplementary Material. In our implementation of biased estimator of HSIC estimator in Equation 2, we use Gaussian RBF kernel function for k(·, ·) and l(·, ·). The training was conducted on a machine with two NVIDIA GeForce RTX 3090, and each training of the U-Net took 1 hour. When the training is complete, the U-net is ready to reconstruct images. Figure 3 shows six examples of how the U-net modifies the original images. We train our model for 5 epochs with an λ = 0.07.

Comparison with baseline models
We compare our results with Ramaswamy et al.'s method, described in their paper 'Fair Attribute Classification through Latent Space De-biasing' [19]. Building on work by [5] which demonstrates a method to learn interpretable image modification directions, they develop an improved method by perturbing latent vector of a GAN, to produce training data that is balanced for each protected attribute. By augmenting the original dataset with the generated data, they train target classifiers on the augmented dataset, and show that these classifiers will be fair, with high accuracy. The second model that we compare our results with is explicit removal of biases from neural network embeddings, presented in [1]. The authors provide an algorithm to remove multiple sources of variation from the feature representation of a network. This is achieved by including secondary branches in a neural network with the aim to minimize a confusion loss, which in turn seeks to change the feature representation of data such that it becomes invariant to the spurious variations that are desired to be removed.
We implement Ramaswamy et al.'s method as follows: As mentioned in their paper, we used progressive GAN with 512-D latent space trained on the CelebA training set from the PyTorch GAN Zoo. We use 10,000 synthetic images and label the synthetic images with a ResNet-18 (modified by adding a fully connected layer with 1,000 neurons). Then we trained a linear SVM to learn the hyper-planes in the latent space as proposed in the original paper. We generate X syn (160,000 images) to generate a synthetic dataset which aims to de-bias Male from all 26 attributes one by one. Next, we train ResNet-18 classifiers on the new datasets consisting of augmenting X and X syn . We call this model as GANDeb. We use the implementation of [1] with the uniform confusion loss −(1/|D|) ∑ d log q d provided in [31].

Evaluation metrics
In evaluating the results of our model with the baseline models, three metrics are used. To capture the accuracy of the classifiers, we measure the average precision. This metric combines precision and recall at every position and computes the average. A higher average precision (AP) is desired. To measure fairness, there are multiple metrics proposed in the literature [15]. Among the most commonly used metrics is demographic parity (DP). This metric captures the disparity of receiving a positive decision among different protected groups (|P(Ŷ = 1|S = 0) − P(Ŷ = 1|S = 1)|). A smaller DP shows a fairer classification and is desired. Finally for our last fairness measure, we follow [14] and [19] and use difference in equality of opportunity (DEO), i.e. the absolute difference between the true positive rates for both gender expressions (|T PR(S = 0) − T PR(S = 1)|). A smaller DEO is desired.

Results
All the values reported in this section, are evaluated on the same test set. Prior to comparing the results of our method with the comparison models, to assess the original training data, the performance of baseline classifiers being trained on the original train set, and tested on the test set is presented. The AP, DP, and DEO values of classifiers trained on the original training set is shown in Table 1 under Baseline. Looking into Baseline values, the AP of classifiers for gender-independent category of attributes is higher than gender-dependent category, and the AP of inconsistent category is less than the other two categories. As expected, DP and DEO for gender-dependent category of attributes is higher than the other two categories. In Table 1, we compare our model with GAN Debiasing (GanDeb) [19], Adversarial debiasing (AdvDb) presented in [1], and the Baseline on the original data. Looking into the average precision scores, the results show that GanDeb is slightly performing better than Ours. This is anticipated, since half of the training data for GanDeb consists of the original images, and therefore a higher average precision is expected. AdvDb on the other hand is performing poorly in terms of average precision, with average precision scores far away from other models.
Looking into demographic parity scores, the results show that GanDeb falls behind the other two models in two out of three attribute categories. While Ours is performing better for gender dependent and gender independent attribute categories. Looking into the third fairness measure, difference in equality of opportunity, AdvDb and ours are performing better than GanDeb in all three categories of attributes. Ours beats AdvDb for inconsistent attributes category, AdvDb beats Ours in gender dependent category, and AdvDb slightly beats Ours for gender independent category of attributes. In summary, Ours is close to GanDeb in terms of maintaining high average precision scores, which means higher accuracy of prediction, while beating GanDeb in terms of fairness metrics. Also, while AdvDb performance in terms of fairness enforcement is better than ours in 3 out of 6 cases, it falls behind significantly in terms of average precision.
To explore the trade-off between fairness and precision, we perform the following experiment: λ was increased between [0.01, 0.15] in steps of 0.01, and for each value of λ , the model was trained three times, each time for 1 epoch. Figure 4 shows how AP, DEO, and DP change. The results show that by increasing λ , precision decreases while fairness measures improve.

Interpretation and the effect on other attributes
In this section, we aim to display the correspondence between an attribute's relationship with Attractive attribute, and the extent to which the model modifies that attribute. To do so, for each attribute, we record two values, namely HSIC value between that attribute and the Attractive attribute, and the change in demographic parity. To calculate the change in demographic parity, we first calculate the demographic parity of the classifier for that specific attribute, when the classifier classifies the original testing set images (similar to Baseline in previous tables, but for each attribute separately). We then calculate the demographic parity of the classifier for that specific attribute, when the classifier receives the modified training images Ours(5,0.07). We then subtract the two values, to get the change in demographic parity for that specific attribute. Figure 5 presents the results, with the red bars showing the change in demographic parity for each attribute, and the blue bars showing the statistical dependence measured by HSIC, between each attribute with Attractive attribute, in the original training data. The results show that the absolute change in demographic parity is positively correlated with that attribute's statistical dependence with the attribute Attractive, with a Pearson correlation coefficient of 0.757. For instance, we observe large changes in demographic parity for attributes such as Young, Big_Nose, Pointy_Nose, Oval_Face, and Arched_Eyebrows, as they are typically associated with being attractive, and therefore reflected in the CelebA dataset labels.

Conclusions
We proposed an image reconstruction process to mitigate bias against a protected attribute. The model's performance was evaluated on CelebA dataset and compared with an augmentation based method developed by [19]. The proposed model showed promising results in mitigating bias while maintaining high precision for classifiers. An interesting aspect of the results is that although we only explicitly train the U-net to remove dependence between the target attribute (Attractive) and the protected attribute (Male), classifiers related to many other attributes, most of which have a statistical dependency with the target attribute, become 'fairer'. An advantage of the proposed model is that it does not rely on modifying downstream classifiers, and rather includes only modifying the input data, hence making it suitable to be deployed in an automated machine learning pipeline more easily and with lower cost. As a potential future direction, we intend to consider the problem in a situation where multiple protected attributes are present, and attributes are non-binary. We also intend to apply similar methodology on other data types such as tabular data.