{'fraction_non_alphanumeric': 0.03614496472097498, 'fraction_numerical': 0.008114175753688262, 'mean_word_length': 4.652828136330674, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
On the effect of age perception biases for real age regression

Automatic age estimation from facial images represents an important task in computer vision. This paper analyses the effect of gender, age, ethnic, makeup and expression attributes of faces as sources of bias to improve deep apparent age prediction. Following recent works where it is shown that apparent age labels benefit real age estimation, rather than direct real to real age regression, our main contribution is the integration, in an end-to-end architecture, of face attributes for apparent age prediction with an additional loss for real age regression. Experimental results on the APPA-REAL dataset indicate the proposed network successfully take advantage of the adopted attributes to improve both apparent and real age estimation. Our model outperformed a state-of-the-art architecture proposed to separately address apparent and real age regression. Finally, we present preliminary results and discussion of a proof of concept application using the proposed model to regress the apparent age of an individual based on the gender of an external observer.


I. INTRODUCTION
Automatic age estimation from still images is intensively studied in Computer Vision [1]- [3] due to its wide range of possible applications, including forensics [4], monitoring and surveillance [5] (e.g., to search for a suspect with a specific age in a database), and recommendation systems [6] (e.g., "how do I perceive to others?"), just to mention a few. Age estimation task requires dealing with several factors such as human variations in appearance or head pose, the use of accessories (e.g., glasses, makeup), hair-style, as well as different illumination conditions, noise and/or occlusion [7]. Moreover, ageing is a variable-paced process depending on each person's genetics and other physiological factors, which make the task even more challenging [2]. For the sake of illustration, few sample images used in this work containing both real and apparent age labels are shown in Fig. 1.
Recent research activities in machine learning (and deep learning) has started to focus on different aspects impacting the outcomes of automatic systems by taking into account subjectivity, human bias perception, fairness, and inclusiveness. In the case of age estimation, recent works started to analyse the apparent age [3] and the perceptual bias involved in age perception [2]. With respect to person perception, recent studies proposed to analyse the biases introduced by observers opinion which are conditioned to facial attributes appearing in a given face image [8], [9]. More interestingly, even with the involved biases, the apparent age labels of face images are proved [2], [3] to achieve better performance for real age regression than training using real age labels. Fig. 1. Samples of the APPA-REAL dataset [3] with respective apparent (Ga) and real (Gr) ground truth. First row: original images. Second row: respective "cropped faces" (provided with the dataset, using [10]). Third row: cropped faces obtained with [11], used in this work.
The present work is inspired by the work of Clapés et al. [2]. They showed that using apparent age labels instead of real ones improves overall real age estimation. Furthermore, they presented preliminary analysis and discussion about the influence of subjective bias in age estimation, i.e., if we can estimate how much an attribute in the data influence/deviates from the target age, then we can correct final estimation and further improve the results. For instance, they postprocessed obtained results and reduced the estimated real ages in a predefined manner after observing people (in general) overestimate female's age.
In this work, we present an end-to-end architecture for real and apparent age estimation which takes into account the way people in the images are perceived by external observers in a multi-task style scenario. We take benefit of apparent age estimation to improve the real age estimation by considering gender, facial expression, happiness and makeup levels 1 . While the end-to-end network in its first layers uses these face attributes as bias to improve apparent age estimation, the last layers of the network are in charge of doing the opposite, i.e., benefiting from an improved apparent age estimation and face attributes to unbias apparent predictions to regress the real age. We show that in practice this works better than directly regressing real age from real age labels (even if face attributes are considered).
Rather than focusing on outperforming the state-of-theart on apparent/real age estimation [3], our main goal is to validate the hypothesis that improvements in both apparent and real age estimation can be tackled jointly in an end-toend fashion when combined with specific attributes people usually use in everyday life when drawing first impressions about others (e.g., gender, ethnic, facial expression). As far as we know, the work of Clapés et al. [2] is the only study proposed to deal with several attributes and apparent age to improve real age estimation.
The contributions of our work are: (i) we provide with an automatic end-to-end approach to improve apparent age prediction and use this together with attributes to also improve real age estimation. Implicitly, it performs the following: in the first part of the network, attributes (bias) are used to improve apparent age estimation and in the second part, from apparent age to real age regression those same attributes are used but in an inverse way, to unbias the apparent age to approximate them to the real age value; (ii) we analyse individually the influence of adopted attributes for real and apparent age estimation; (iii) we outperformed the baseline model [2] by using a more robust architecture trained endto-end, rather than using an add-hoc post-processing stage; and (iv) we present preliminary results and discussion when the gender attribute of people who label the data is also taken into account. This way the model is able to regress the apparent age of an individual based on the gender of an external observer.
The rest of the paper is organised as follows: Section II presents the related work on real and apparent age estimation, with a particular focus on face attributes analysis as a source of bias for age estimation. The proposed model and experimental results are presented in Sections III and IV, respectively. Finally, our conclusions are drawn in Section V.

II. RELATED WORK
This section reviews related work on real and apparent age estimation. Early and recent works are briefly discussed without the intention of providing an extended and comprehensive review on the topic. To this end, we refer the reader to [1]. Then, we revisit related studies on the analysis of bias in age estimation.

A. Real Age Estimation
Early works on real age estimation are based on handcrafted approaches. Lanitis et al. [12] proposed to use active appearance models to define compact feature spaces which are regressed for real age estimation. Yun et al. [13] based on manifold analysis and multiple linear regression functions for real age estimation. Guodong et al. [7] analysed age estimation using a pyramid of Gabor filters.
Recent works for real age estimation in images and videos benefit from the advances in deep learning and end-to-end architectures. For instance, Pei et al. [14] proposed an end-toend architecture for learning the real age of given facial video sequence. The model is based on the extraction of latent appearance representations which are learnt by a Recurrent Neural Network (RNN). González-Briones et al. [15] proposed an ensemble of age and gender recognition techniques, showing the benefits of late fusion from independent learners.

B. Apparent age estimation
In the case of apparent age estimation, each face image usually contains multiple age labels, related to variations in perception coming from different annotators/observers. Agustsson et al. [3] reported that real age estimation could be successfully tackled as a combination of apparent and real age estimation by learning residuals. Geng et al. [16] modeled an aging pattern by constructing a subspace given a set of ordered face images by age. In the aging pattern, each position indicates its apparent age. Zhu et al. [17] proposed to learn deep representations in a cascaded way. They analysed how to utilise a large number of face images without apparent age labels to learn a face representation, as well as how to tune a deep network using a limited number of labelled samples. Malli et al. [18] proposed to group face images within a specified age range to train an ensemble of deep learning models. The outputs of these trained models are then combined to obtain a final apparent age estimation.

C. Analysis of bias in age estimation
While state-of-the-art machine learning algorithms can provide accurate prediction performances for age estimation, either if real or apparent age are considered, they are still affected by different variations in face characteristics. But how can age prediction performances be enhanced in this case? With this objective in mind, the analysis of bias in age perception has recently emerged [2], [9]. Can we better understand age perception and their biases so that the findings can be used to regress a better real age estimation? In this line, Clapés et al. [2] found some consistent biases in the APPA-REAL [3] dataset when relating apparent to real age. However, an end-to-end approach for bias removal was not considered. According to Alvi et al. [9], training an age predictor on a dataset that is not balanced for gender can lead to gender biased predictions. They presented an algorithm to remove biases from the feature representation, as well as to ensure that the network is blind to a known bias in the dataset. Thus, improving classification accuracy, particularly when training networks on extremely biased datasets.
Differently from previous works, our aim is to explicitly use face attributes as sources of biases to regress a more accurate apparent age. Then, to unbias perceived age for real age regression. As we show later, taking benefit of biases in age perception can improve accuracy of real age estimation.

III. PROPOSED MODEL
This section describes the proposed end-to-end architecture to jointly predict apparent and real age from facial images. The proposed architecture consists of two main stages. At the first stage, our goal is to better approximate the human perception mechanism by introducing human perception bias when predicting apparent ages using face attributes. At the second stage, we aim to remove bias when predicting the real age. The proposed model combines apparent and real age labels with additional face attributes (i.e., gender, race, level of happiness, and makeup) during training. Note that, once the model is trained, it uses neither real nor apparent age labels on the test set.
To achieve our objectives, and deal with the problem of jointly estimating apparent and real ages, we modified the VGG16 model [19], which was pre-trained on ImageNet. The VGG16 model is a high capacity network utilised in most face analysis benchmarks. We modify the last layers of this base model in a way to reduce the bias of the apparent age estimation and accurately estimate the apparent and real age. Figs. 2 and 3 illustrate the VGG16 base model with our modifications, which are explained next.

A. Apparent age estimation
To introduce human bias into the model, different attributes people use to perceive others are considered, which may affect the perceived age. They are gender (male, female), race (Asian, Afroamerican and Caucasian), level of happiness (happy, slightly happy, neutral and other) and makeup (makeup, no makeup, not clear and very subtle makeup). Note that state-of-the-art methods could be used to accurately recognise such attributes from face images (e.g., [6], [15], [20], [21]). However, as the focus of our work is not on improving the recognition accuracy of such attributes, and because of the required amount of data to learn those associated tasks accurately, we decided to import them directly from the adopted dataset [2].
For the sake of simplicity, first consider the convolutional layer highlighted by a blue box in Fig. 2 and Fig. 3 (block5 pool) represent the same layer in both models, the base VGG16 and the proposed one. This illustrates which layers have been removed and which ones have been introduced. Such convolutional layer has high dimensionality, in particular if we consider the idea of combining with it a very low dimensional vector composed of people's attributes. In this work, each different attribute is encoded using one hot vector, usually employed to represent categorical data (e.g., male = [0, 1] and female = [1, 0]). All considered attributes are then concatenated, resulting in a vector (input 2, in Fig. 3) of length 13, which is further encoded in a dense layer (hidden layer, 10D). In order to reduce the dimensionality of the previously mentioned high dimensional layer, a new (7 × 7 × 512) convolutional layer, followed by ReLU, is included just after it. The resulting (flatten 1) layer is then concatenated with people's attributes. Finally, a new FC layer (f c2 = 256D) is responsible to fuse both information before regress the apparent age, using a Sigmoid function.
With these network updates, we expect it to benefit from people's attributes to better approximate apparent age rather than just considering raw input image.

B. Real age estimation
In the previous section we described how we perform apparent age estimation from visual information and people's attributes. As mentioned before, recent works [2], [3] reported that real age estimation could be better approximated when apparent age labels are used rather than real age ones. The main hypothesis behind such idea is that it could somehow help the network by reducing the "noise" in the label space, and in particular of those people with a real age that highly differ to their apparent one.
In this work, instead of applying a post-processing bias correction scheme based on statistical analysis as in [2], we propose to correct such human perception bias during training. To achieve this goal, the predicted apparent age (represented by a "salmon" box in Fig. 3) is first concatenated with face perceptual attributes using a different representation (hidden layer 2, 5D). Then, the concatenated information is fused in a FC layer (f c3 = 6D) before regressing the real age. This way, we expect real age estimation can benefit from both, apparent age estimation and people attributes.

C. Training strategy
We perform the training in two stages. In the first stage, just the included layers are fine-tuned, i.e., those shown in Fig. 3). In a second stage, the whole network is trained in an end-to-end manner. Adam algorithm is used as an optimisation method with default values, except the learning rate (lr), which was set based on the results obtained on validation. The mean squared error is defined as a loss function for both apparent and real age estimation, with both losses having equal weight.
Each case study presented in Sec. IV had its associated model trained using a train/validation sets and evaluated in a complete and different test set, all provided with the adopted APPA-REAL [3] database. We perform early stopping based on the validation set loss. The maximum number of epochs was set to 3000 and 1500, for stage 1 and 2, respectively. However, in most of the cases (except for Case 1, described next) the training stopped before achieving such limits. Anyway, no significant improvements have been observed on Case 1 with higher iterations. As stop criteria for predicting apparent age, the Mean Absolute Error (MAE) with respect to apparent age labels was adopted, whereas when predicting the real age the stop criteria considered the loss with respect to real age labels. Finally, the best model is kept based on the accuracy computed on the validation set.

IV. EXPERIMENTAL RESULTS
This section describes experimental results obtained in three case studies. The first case study (Sec. IV-B) is based on the original VGG16 model, with the final layer modified to regress either apparent or real age. The second case study (Sec. IV-C) illustrates the benefit of including people's attributes to the visual stream using a fraction of the proposed model, i.e., ignoring those dashed connections in Fig. 3 and predicting either the apparent or the real age, based on the given input. The third case study (Sec. IV-D) shows Fig. 2. VGG16 base model. Layers highlighted in red have been modified to jointly predict apparent and real ages, as detailed in Fig. 3.   Fig. 3. Overview of the proposed model. It uses the same structure as the VGG16 base model from the input 1 to the last convolutional layer (highlighted in blue, also shown in Fig. 2). Then, new layers are include to deal with people's attributes such as gender, race, happiness and makeup level (input 2, represented by a yellow box) before regressing the apparent age ("salmon" box). The predicted apparent age is then used to support the final real age estimation (green box). The dashed arrows illustrates the connections responsible to fuse both apparent and real age information. The dimension of new layers are shown below/above them. experimental results obtained using the proposed end-to-end architecture to jointly estimate apparent and real age. This way, we show improvements obtained with the inclusion of new features and more complex models incrementally. Then, a discussion on the results based on the considered people attributes is presented in Sec. IV-E. Finally, we show a proof of concept application using the proposed model to regress apparent age based on the gender of an external observer (Sec. IV-F). Next, we briefly describe the adopted dataset and evaluation protocol.

A. Dataset and evaluation protocol
The APPA-REAL [3] database was extended in [2] with the inclusion of few attributes about people appearing in the images, i.e., gender (male, female), race (Asian, Afro-American and Caucasian), happiness (happy, slightly happy, neutral and other) and makeup category (makeup, no makeup, not clear and very subtle makeup). They also provide for a subset of the dataset the gender of people who labelled the data (used in Sec. IV-F). The dataset is composed by almost 8K images of (mostly) single persons in frontal faces. However, it also includes images of full body, upper body or an expanded region of the face, with different resolutions (black and white or coloured images), different image qualities and severe head pose variation and occlusion problems. In some occasions there are multiple faces (mainly on the background), making the task even more challenging. The dataset is also provided with cropped and aligned faces, captured using [10]. However, some false positive samples, as well as wrongly detected faces, are still present after such procedure. To minimise the face detection related problems, we adopt a more robust face detection/alignment algorithm [11]. A qualitative comparison about both approaches is illustrated in Fig. 1.
To quantitatively evaluate the results, we followed the evaluation protocol defined in [2], [3], i.e., computing the Mean Absolute Error (MAE) between the ground truth and predicted ages on the test set.

B. Case 1: VGG16 baseline
In this case study, the VGG base model is used as it is shown in Fig. 2. However, the last layer is modified to regress one single value using a Sigmoid function. The model is trained as described in Sec. III-C, using lr 1 = 1e − 6.
Obtained results for different inputs/outputs are shown in Table I, top. As it can be seen, real age estimation was better predicted from apparent age labels than from real ones, which is aligned with results obtained in [2] (see Table II). It emphasises the point that human bias introduced by external observers can be used to improve real age estimation. Next, we show how the inclusion of people's attributes can further improve these results.

C. Case 2: (extended) VGG16 with face attributes
In this experiment, just a fraction of the proposed model is used. Concretely, those layers connected by dashed lines in Fig. 3 have not been considered. Then, instead of predicting apparent age on the predict app layer, and refining the real age from its output, we simply predicted either apparent or real age, according to the desired goal and respective input, i.e., apparent or real age labels. The model is trained as described in Sec. III-C, using lr 1 = 1e − 4. Obtained results for different inputs/outputs are shown in Table I, bottom. As it can be seen, the inclusion of people's attributes helped to improve all scenarios (compared to case 1), aligned with the assumption that the way people perceive others is strongly influenced by gender [22], facial expression [23], among other attributes. Indeed, this way real age estimation was able to benefit from both human biases introduced by external observers and people's attributes.
To verify if the improvements obtained by the inclusion of people's attributes (i.e., case 2) were not influenced by the model architecture, compared to "case 1", we considered an additional experiment where "case 2" did not take into account people's attributes, i.e., the input 2 layer shown in Fig. 3 has not been considered (as well as its outputs). Obtained results, referred in Table I as case 2', show that the inclusion of people's attributes still slightly benefit both apparent and real age estimation.

D. Case 3: proposed model
In this case study, we report the results obtained using the (complete) proposed model 2 shown in Fig 3. The model is trained as described in Sec. III-C, using lr 1 = 1e − 4.
Obtained results are shown in Table II, as well as those reported in [2]. Note that our main goal is not to outperform the state-of-the-art in apparent/real age estimation, but to predict apparent and real ages jointly considering people's attributes in an end-to-end fashion. Thus, benefiting from human bias during training. As it can be seen, the proposed model outperformed [2] and further improved apparent and real age estimation compared to previous cases 1 and 2, indicating that tackling the problem jointly benefit both apparent and real age regression tasks.  Table III shows the number of trainable parameters for each model used in previous experiments. As it can be seen, the proposed model was able to achieve better results than VGG16 baseline (Case 1) using a significantly smaller number of trainable parameters.  Fig. 4 shows the average error, with respect to real age estimation, for different age ranges obtained from different inputs and case studies (computed considering a window of five years to facilitate visualisation). As it can be seen, real age estimation overall benefited from people's attributes and apparent labels when different age ranges are considered. The proposed model (Case 3) obtained similar or better results for real age estimation compared to Case 2, indicating that the problem can benefit from both tasks (real and apparent age estimation) when jointly analysed. In Fig. 5 a similar plot is shown but for apparent age estimation. As it can be seen, the (complete) proposed model did not improve its partial version (Case 2) for some age ranges (e.g., higher than ∼ 50) with respect to apparent age estimation. This may be due to the fact that the proposed model was optimised (during training) to improve both real and apparent age estimation, while other cases were dedicated to optimising the apparent age only. In both scenarios ( Fig. 4 and Fig. 5), the curves show lower error values for those ages with higher number of samples on the train set (Fig. 6).    Fig. 7 shows qualitative results obtained using the proposed model, with images sorted based on real age estimation error. Additional qualitative results are shown in Fig. 8, in this case with images sorted based on apparent age estimation error. In Fig. 9 we show few examples of unsatisfactory results for both cases, which may be caused due to partial occlusion, illumination condition, head-pose or even due small number of samples in the train set for those age ranges.

E. Attribute-based analysis
In Table IV we show computed errors for each attribute and case study with respect to real and apparent age estimation. Results reported for Cases 1 and 2 do not use real age labels (i.e., the predicted real ages were based on apparent age labels). The percentile of each category on the train set is shown to illustrate the distribution of each attribute and the presence (or not) of unbalanced data. Note that all attributes have been analysed individually in order to make the analysis simple so that few and consistent observations could be taken. A higher level analysis could consider the intersection among different attributes, but probably would require a larger (and balanced) dataset.
From Table IV we can make the following observations: • Gender: real age is slightly better predicted for males in all cases, even though using a balanced train set. Regarding apparent age, obtained results are similar, with a slight error decrease for females. Unsatisfactory results obtained using the proposed model. G = ground truth, P = prediction, (a|r) = apparent or real age.
• Race: the train set for this attribute is strongly unbalanced. Taking real age estimation, in general, Afro-Americans obtained the highest real age estimation error, followed by Caucasians and Asians. Clearly, the small number of samples in the Afro-American category is the main reason for these results. Interestingly, Asians obtained the lowest real age error even when having less than 11% of samples for training. Regarding apparent age, Afro-Americans obtained overall lowest error rates even with less than 3% of samples on the train set.
On the other hand, Caucasians obtained the highest error rates when having more than 86% of samples for training. These results indicate that the perception of age of Asian and Afro-American people is more accurate if compared to Caucasians, which may reflect some physiological phenomena. For instance, it was  [25], i.e., neutral faces are more easily estimated, whereas age of happy faces tends to be underestimated. • Makeup: even though having an unbalanced dataset for this attribute, no significant difference was observed for real age estimation (i.e., if "not clear" class is ignored, as it has less than 1% of train data). In general, the absence of makeup (or very subtle makeup) helped to achieve lowest error rates for real age estimation. On the other hand, the presence of makeup helped to improve results for apparent age estimation.
F. How your age is perceived based on the gender of the observer?
This section presents a proof of concept application to regress apparent age based on the input gender of a given observer. For this, we base on the gender information of labellers provided for a subset of the APPA-REAL dataset [2]. Thus, the model can be trained considering gender observer information as an additional input variable to previously considered attributes. The main aim of this application is to automatically respond the question: how a male/female will perceive the age of the person in this image? Note that higher analysis considering the intersection among all attributes (observer vs. observed people) could be considered as future work.
For a subset of individuals on the train/validation sets, the gender of the observer is provided in addition to the respective apparent age label. For the test set, all samples contain annotations from different observers (of both genders, male/female). Then, for each sample on the train/validation/test set, we computed the average apparent age with respect to these two points of views, i.e., male/female observer. Thus, we have for each individual his/her average apparent age from two observer genders (which composes the ground truth of this "new dataset"). It resulted in a small train set (compared to the original one) but large enough to report some initial results. In total, the new train set is composed of 1328 samples and 504 validation samples. Fig. 10 shows the distribution of this new train set.
We modified the input 2 layer of the proposed model (shown in Fig. 3) to include the gender of the observer, encoded as one hot vector and concatenated with the attributes of the person being observed. It resulted in a vector of length 15 (instead of the original 13). The model is trained as described in Sec. III-C, using lr 1 = 1e − 4.
Obtained results indicate that the inclusion of the gender of the observer did not have a strong impact on the outcomes, as the computed error (MAE) for both points of views (males/females) are somehow similar, i.e., 9.758 for female observers and 9.243 for males. However, these preliminary results suggest that the male's perception was modelled with slightly better accuracy. Note that these results cannot be compared to those reported before since data and annotations have been modified. Some qualitative results obtained by the model are shown in Fig. 11. Note that although differences in apparent ground truth for males and females do not highly differ, predictions of our model when input gender varies provides a closer apparent prediction consistent to the gender of the observer.

V. CONCLUSION
In this work, we proposed an end-to-end CNN architecture for real age estimation that benefits from apparent age information. The network in a first stage uses face (a) G f |m ={51.4 | 47.3} P f |m ={51.0 | 50.54} (b) G f |m ={29.1 | 27.4} P f |m ={27.1 | 26.7} Fig. 11. Results obtained using the proposed model when taking into account the gender of the observer. G = apparent age ground truth, P = apparent age prediction, (f |m) = female/male observer. attributes in target image as input variables to learn their age perception biases and regress the apparent age. The second part of the network benefits from both input attributes of face and apparent age prediction to regress a final real age prediction, like performing unbias. We showed in the APPA-REAL dataset that proposed network, integrating both apparent and real age predictions, achieves better recognition for both tasks that when they are addressed separately. We also provided with a proof of concept application where the network was trained including the gender of the age guessers. This way, during testing we could retrieve the apparent age for an input image given the gender of an observer.
Future work will include the extension of both amount of data and number of attributes for a deeper analysis of the bias involved in age perception. We plan to jointly recognise those attributes together with apparent and real age tasks for a fine-grain analysis on the problem, allowing the multi-task network to share weights from early training stages among all tasks.