{'fraction_non_alphanumeric': 0.027084098420859345, 'fraction_numerical': 0.006472640470069776, 'mean_word_length': 4.894209956709957, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
New approach using Bayesian Network to improve content based image classification systems

This paper proposes a new approach based on augmented naive Bayes for image classification. Initially, each image is cutting in a whole of blocks. For each block, we compute a vector of descriptors. Then, we propose to carry out a classification of the vectors of descriptors to build a vector of labels for each image. Finally, we propose three variants of Bayesian Networks such as Naive Bayesian Network (NB), Tree Augmented Naive Bayes (TAN) and Forest Augmented Naive Bayes (FAN) to classify the image using the vector of labels. The results showed a marked improvement over the FAN, NB and TAN.


Introduction
In the few last years, the domain of indexation by the content called CBIR (Content Based Image Retrieval), which includes all that works developed to search and classify images through their internal content, has been a very active domain of research. Much of the related work on image recognition and classification for indexing, classifying and retrieval has focused on the definition of low-level descriptors and the generation of metrics in the descriptor space [1]. These descriptors are extremely useful in some generic image classification tasks or when classification is based on query by example. However, if the aim is to classify the image using the descriptors of the object content this image. There are two questions to be answered in order to solve difficulties that are hampering the progress of research in this direction. First, how shall we semantically link objects in images with high-level features? That means how to learn the dependence between objects that reflect better the data? Second, how shall we classify the image using the structure of dependence finding?
Our paper presents a work which uses three variants of naïve Bayesian Networks to classify image of faces, using the structure of dependence found between objects. This paper is divided as follows: Section 2 presents the principal works related to CBIR and a state of art of 2D face recognition; Section 3 describes the developed approach based in Naïve Bayesian Network, we describe how the spatial features are extracted; and we introduce the method of building the Naïve Bayesian Network, Tree Augmented Naïve Bayes (TAN) and Forest Augmented Naïve Bayes (FAN), and inferring posterior probabilities out of the network; Section 4 presents some experiments; finally, Section 5 presents the discussion and conclusions.

State of the art in 2D Face Recognition
Face recognition is a huge research area and is the preferred mode of identity recognition by humans: it is natural, robust. Each year, the attempted solutions and algorithm grow in complexity and execution time. Zhao et al. in [28] divide the existing algorithms into three categories, depending on the information they use to perform the classification: appearance-based methods (also called holistic), feature-based and hybrid. Sharavanan et al. in [42] present taxonomy of face recognition methods used as shown in the figure 1. In this section, we review the relevant work on 2D face recognition, and discuss the merits of different representations and recognition algorithms.

Holistic Approach
Holistic algorithms use global features of the complete face. Examples of holistic systems include Principal Component Analysis (PCA) [31], Linear Discriminant Analysis (LDA) [33] as well as Independent Component Analysis (ICA) [32]. These projection techniques are used to represent face images as a lower-dimensional vector, and the classification itself is actually performed by comparing these vectors according to a metric in the subspace domain.

 Principal Component Analysis
Principal Component Analysis (PCA) is an approach to analyze and dimensionally reduce the data, highlighting their similarities and differences. The task of facial recognition is discriminating input signals (image data) into several classes (persons).

 Linear Discriminate Analysis
Linear Discriminate Analysis is effective to encode discriminatory information. The idea of LDA is grouping similar classes of data where as PCA works directly on data. It tries to find directions along which the classes are best separated by taking into consideration not only the dispersion within-classes but also the dispersion between classes.

 Independent Component Analysis
Independent component analysis (ICA) is a generative model, expressing how observed signals are mixed from underlying independent sources. This approach transforms the observed vector into components which are statically maintained as independent. The basic restriction is that the independent components must be non-Gaussian for Independent Component Analysis to be possible. A limitation of holistic matching is that it requires accurate normalization of the faces according to position, illumination and scale. Variations in these factors can affect the global features of the face, leading inaccurate final recognition. Moreover, global features are also sensitive to facial expressions and occlusions. Holistic methods such as neural networks [24] are more complex to implement, whereby an entire image segment can be reduced to a few key values for comparison with other stored key values with no exact measures or knowledge such as eye locations.

Feature-based Approach
Feature-based algorithms use local features [26] or regions [27] of the face for recognition. Such features may be geometric measurements, particular blocks of pixels, or local responses to a set of filters. Feature-based algorithms try to derive an individual's face's model based on local observations. Examples of such systems include the Elastic Bunch Graph Matching (EBMG) [38], recent systems using Local Binary Patterns (LBP) [34] [37], and also statistical generative models: Gaussian Mixture Models (GMM) [35] [36].

 Elastic Bunch Graph Matching
This technique use the structure information of a face, which shows the fact that the images of the same subject tend to deform, scale, translate, and rotate, in the plan of image. It uses the labeled graph, edges are labeled according to the distance information and nodes are labeled with wavelet coefficients in jets. This graph can be after used to produce image graph. The model graph can be deformed, scaled, translated, and rotated during the matching process. This can make the system resistant to large variation in the images.

 Dynamic Link Matching
In dynamic link structure, the models are presented by layers of neurons, which are labeled by jets. Jets are wavelet elements describing grey level distribution. This approach encodes information using wavelet transformations. Dynamic link structure establishes a mapping between all-in-all linked layers, and thus reduces distortion. The method uses a winner-take-all strategy once a correct mapping is obtained choosing the right model. A typical dynamic link structure is given in the figure 2.  [42] Face recognition systems using local features empirically show a better performance as compared to holistic algorithms [40] [36]. Moreover, they also have several other advantages: first, face images are not required to be precisely aligned. This is an important property, since it increases robustness against imprecisely located faces, which is a desirable behavior in real-world scenarios. Second, local features are also less sensitive to little variations in illumination, scale, expressions and occlusions. To go beyond the limitations of the Eigen faces method in holistic methods, Garcia et al. in [30] present a new scheme for face recognition using wavelet packet decomposition. Each face is described by a subset of band filtered images containing wavelet coefficients. These coefficients characterize the face texture, and a set of simple statistical measures permits forming compact and meaningful feature vectors. Then, an efficient and reliable probabilistic metric derived from Bhattacharrya distance is used in order to classify the face feature vectors into person classes. The Wavelet transformation allows the generation of such facial characteristics that are invariant to lighting conditions and overlapping. It organizes the image in subgroups that are localized according to orientation and frequency. In every subgroup, each coefficient is also localized in space.

Hybrid Approach
Hybrid matching methods use a combination of global and local-features for recognition [33]. This approach is most effective and efficient in the presence of irrerelevant data.
The key factors affecting performance depend on the selected features and the techniques used to combine them. Feature based and holistic methods are not devoid of drawbacks. Feature based method is sometimes badly affected by accuracy problem since accurate feature localization is its very significant step. On the other hand, holistic approach uses more complex algorithms demanding longer training time and storage requirements. Huang et al in [41] proposed a hybrid face recognition method that combines holistic and feature analysis-based approaches using a Markov Random Field (MRF) model. According to the proposed method, the image is divided into smaller image patches, each having specific Ids. The model can be represented as in the figure 3. It includes two layers, observable nodes (squares in the graph, representing image patches) and hidden nodes (circles in the graph, representing the patch Ids). Edges in the graph depict relationships among the nodes.

State of the art in Bayesian classification
There are several works in CBIR systems using Bayesian Network. The majority of them involve various small problems, such as structure of Bayesian Network, feature extraction, classification and retrieval. Rodrigues in [3] presents a Bayesian network model which combines primitives' information of color, edge-map and texture to improve the precision in content-based image retrieval systems. This approach allows retrieving images according to their features. However the spatial relations between objects are not treated. Zhang in [2] suggested modeling a global knowledge network by treating and entire image as a scenario. Authors suggested a process that is divided into two stages: the initial retrieval stage and Bayesian belief network inference stage. The initial retrieval stage is focused on finding the best multi-features space, and then a simple initial retrieval is done within this stage. The Bayesian inference stage models initial beliefs on probability distributions of concepts from the initial retrieval information and construct a Bayesian belief network. In this approach, the results are largely depending on the choice of representative blocks in the first stage. However, the dependence between objects in the approach presented in [2] is also not dealt with. Mahjoub and Jayech in [4] investigated three variants of Bayesian Networks such as Naïve Bayes, Tree Augmented Naïve Bayes (TAN) and Forest Augmented Naive Bayes (FAN) in image classification. Those models study the dependence between objects. However, the images used in this study represent the structure of a document containing texts blocks and graphs. The work presented in this paper tries to accomplish the difficult task of classifying the images of faces using the high level features. Our works study the dependence between the high level features to build an optimal structure that better represents the data saying if an object IJCSI International Journal of Computer Science Issues, Vol. 7, Issue 6, November 2010 ISSN (Online): 1694-0814 www.IJCSI.org 56 is depends or not to another by using three variants of naïve Bayesian Networks.

The proposed Approach
Classifying an image depends on classifying the objects within the image. The studied objects in an image are the following analysis represented by blocks. An example of one image being divided into elementary building blocks that contain different concepts is illustrated in figure 4: For each block, we compute the histogram of color using GMM (Gaussian Mixture Model) and we compute the descriptor textural using the Grey Level Co-occurrence Matrix (GLCM). Then, we use Kmeans to cluster the object into k cluster. So each block will be labeled and integrated to one of the k cluster The estimation of these parameters can be doing with the EM proposed by Dempster and al in 1977 and the number of k normal component can be estimated as follows: The value of k is chosen between k=1 and k (k is to choose in priori) that minimizes the Bayesian criterion BIC [7]. θ K and v K are respectively the maximum credibility estimator and the degree of freedom of the model. Textural features are linked to the contrast of processed images. The textural features can be used to differentiate between objects having the same color. Among these features, we used the Haralick features [5]. Haralick features' computing is based on co-occurrence matrix. From these features, we use four attributes which are: energy, entropy, contrast, and homogeneity. The energy is defined as: The entropy is defined as ENT p i, j log p i, j The contrast is defined as The homogeneity is defined as

Clustering with K-means
The attributes of this work is the clustering of vectors descriptors of the images to improve the classification by Bayesian Network by optimizing its construction.
For each object extracted of the image, we compute the two features describing a given object: color and textural features. The feature attributes are calculated for each image of the data-base and then clustered into k cluster. We use the method of k-means to cluster the descriptor as shown in figure 6.
 Fig. 6 The result of clustering with k-means into five clusters

Classifiers and learning
In this study, we utilize Naïve Bayes, Tree Augmented Naïve Bayes (TAN) and Forest Augmented Naïve Bayes (FAN) classifiers. This sub-section describes the implementation of these methods

Structure learning  Naïve Bayes Network (NB)
A variant of Bayesian Network is called Naïve Bayes. It is one of the most effective and efficient classification algorithms. Figure 7 shows graphically the structure of naïve Bayes, each attribute node has the class node as it parent, but does not have any parent from attribute node.
As the values of | can be easily calculated from training instances, naïve Bayes is easy to construct. It is obvious that the conditional independence assumption in naïve Bayes is in fact rarely true. Indeed, naive Bayes has been found to work poorly for regression problems (Frank et al., 2000), and produces poor probability estimation (Bennett, 2000). One way to alleviate the conditional independence assumption is to extend the structure of naive Bayes to explicitly represent attribute dependencies by adding arcs between attributes. Tree augmented naive Bayes (TAN) is an extended treelike naive Bayes (Friedman et al., 1997), in which the class node directly points to all attribute nodes and an attribute node can have only one parent from another attribute node (in addition to the class node). Figure 8 shows an example of TAN. In TAN, each node has at most two parents (one is the class node). The experiments show that classification accuracy in TAN is higher than in Naïve Bayes. Two factors contribute to this fact. First, some edges may be unnecessary to exist in a TAN. The number of the edges is fixed to n-1. Sometimes, it might outfit the data. Second, in step 4, the root of the tree is randomly chosen and the directions of all edges are set outward from it. However, the selection of the root attribute is important because it defines the structure of the resulting TAN, since a TAN is a directed graph. It is interesting that the directions of edges in a TAN do not significantly alter modify the classification accuracy. So, we correspondingly modify the TAN algorithm. First, we remove the edges that have conditional mutual information less than a threshold. To our understanding, these edges have a high risk to outfit the training data, and thus undermine the probability estimation. More precisely, we use the average conditional mutual information Iavg, defined in Eq.(1), as the threshold.
Where n is the number of attributes. Second, we choose the attribute A with the maximum mutual information with class, defined by Eq.(2), as the root: A argmax A I A ; C (2) Where i=1,…,n. It is natural to use this strategy, since intuitively the attribute which has the greatest influence on classification should be the root of the tree Since the structure of the resulting model is not a strict tree, we call our algorithm Forest augmented naive Bayes (FAN), described in detail as follows.  that has the maximum mutual information with class, as follows: A argmax A I A ; C 5: Transform the resulting undirected tree into a directed one by setting A as the root and setting the direction of all edges to be outward from it. 6: Delete the directed edges with the weight of the conditional mutual information below the average conditional mutual informationIavg. 7: Construct a FAN model by adding a vertex labeled by C and adding a directed arc from C to eachA .

Parameters learning
NB, TAN and FAN classifiers parameters were obtained by using the procedure as follows (Ben Amor 2006). In the implementation of NB, TAN and FAN, we used the Laplace estimation to avoid the zero-frequency problem. More precisely, we estimated the probabilities P c , P a |c and P a |a , c using Laplace estimation as follows. -N c, a , a : is the number of instances in class c and with A a andA a .

Classification
In this work the decisions are inferred using Bayesian Networks. Class of an example is decided by calculating posterior probabilities of classes using Bayes rule. This is described for both classifiers.

 NB classifier
In NB classifier, class variable maximizing Eq.(3) is assigned to a given example.
P C|A P C P A|C P C ∏ P A |C

Experiments and Results
Now, we present the results of the contribution of our approach to classify images of some examples of classes from the database used 'Database of Faces'. The ORL face database consists of ten different images of each of 40 distinct subjects with 92*112 pixels. For some subjects, the images were taken at different times, varying the lighting, facial expressions (open /closed eyes, smiling / not smiling) and facial details (glasses / no glasses) as shown at figure 9. All the images were taken against a dark homogeneous background with the subjects in an upright, frontal position (with tolerance for some side movement).

Parameter learning
We estimated the conditional and a priori probability P c , P a |c and P a |a , c using Laplace estimation. We have used the Laplace estimation described in section (3.3.2) to avoid the zero-frequency problem; we obtained the results as follows:

Results
For each experiment, we used the percentage of correct classification (PCC) to evaluate the classification accuracy defined as: PCC number of images correctly classified total number of images classified Other criterion can be used, such as:  P X C1 | class C1 and P X C2 | class C2 represent the rate of good classification.  P(X=C1 | class=C2) and P(X=C2 | class=C1) represent the rate of bad classification.
The results of experiments are summarized in Table 3 and Table 4 with different number of clusters k=5 and k=10. Naive Bayes, TAN and FAN use the same training set and are exactly evaluated on the same test set. As shown in experiments results in Table 3, the Naïve Bayes performed better than Tree Augmented Naïve Bayes (TAN) and Forest Augmented Naïve Bayes (FAN). However, when we increase the threshold in FAN we find out that the rate of mean classification is improved and became better than NB and TAN.  As shown in experiments results in Table 4, the classification accuracy with number of cluster equal to 10 performed better than classification accuracy with number of cluster equal to 5. So, we were obliged to study the effect of variation of number of cluster in the classification accuracy. As experiments results shown in figure 15, we see that the mean of classification accuracy saved by Naïve Bayes is higher when number of cluster k=8 and then it decreased. So, we concluded that number of cluster is optimal when k=8 that means the number of cluster in this case reflect better the data.

Conclusion
In this study we have presented a new approach for classifying image of faces by using Bayesian Network and Kmeans to cluster the vectors descriptors of the images to improve the classification using Bayesian Network by optimizing its construction. We have implemented and compared three classifiers: NB, TAN and FAN. The goal was to be able to apply algorithms that can produce useful information from a high dimensional data. In particular, the aim is to improve Naïve Bayes by removing some of the unwarranted independence relations among features and hence we extend Naïve Bayes structure shown in figure 7 by implementing the Tree Augmented Naïve Bayes. Unfortunately, our experiments show that TAN performs even worse than Naïve Bayes in classification. Responding to this problem, we have modified the traditional TAN learning algorithm by implementing a new learning algorithm, called Forest Augmented Naïve Bayes. We experimentally test our algorithm in data image of faces and compared it to NB and TAN. The experimental results show that FAN improves significantly NB classifiers' performance in classification. In addition, the results show that the mean of classification accuracy is better when the number of cluster is optimal that means the number of cluster that can reflect better the data.