{'fraction_non_alphanumeric': 0.031000293968304872, 'fraction_numerical': 0.012988054197065661, 'mean_word_length': 4.906866614048934, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DFC: A Performant Dagging Approach of Classification Based on Formal Concept

Knowledge discovery data (KDD) is a research theme evolving to exploit a large data set collected every day from various fields of computing applications. The underlying idea is to extract hidden knowledge from a data set. It includes several tasks that form a process, such as data mining. Classification and clustering are data mining techniques. Several approaches were proposed in classification such as induction of decision trees, Bayes net, support vector machine, and formal concept analysis (FCA). The choice of FCA could be explained by its ability to extract hidden knowledge. Recently, researchers have been interested in the ensemble methods (sequential/parallel) to combine a set of classifiers. The combination of classifiers is made by a vote technique. There has been little focus on FCA in the context of ensemble learning. This paper presents a new approach to building a single part of the lattice with best possible concepts. This approach is based on parallel ensemble learning. It improves the state-of-the-art methods based on FCA since it handles more voluminous data.

richness is guaranteed by the Formal Concept Analysis (Poelmans et al., 2013). Unfortunately, this classification methods encountered some problems such as an exponential complexity (in the worth case), a high error rate and overfitting (Meddouri & Maddouri, 2009;Meddouri & Maddouri, 2010).
Several ensemble methods are used to improve the error rate of any single learner (Freund, 1995) (Freund & Shapire, 1996). These proposed methods are based on sequential learning (Boosting). All the data are considered in each learning step and the weights are assigned to learning instances. However, Kuncheva reported that sequential learning (Boosting) is not enough for efficient classifier such as Decision Tree (Kuncheva et al., 2002). In the area of supervised learning, other ensembles exist, and they are based on parallel learning. The difference between these two ensemble methods derives from how to select data for learning. They are distinguished by the data sampling techniques as Bootstrapping used to learn the classifiers from subsets. The particularity of learning from a Bootstrap is to combine hard learning instances to misleading instances in the training set (unlike the sequential approach) (Breiman, 1996;Breiman, 1996b;Kuncheva, 2004).
There has been little research that focused on the classification based on Formal Concepts Analysis as part of the parallel learning. We propose to use and study the Formal Concepts Analysis in this context and compare it with respect to the sequential approach. The best-known method, which is based parallel learning is Dagging (Disjoint samples aggregating) creates a number of disjoint groups and stratified data from the original learning data set (Ting & Witten, 1997), each considered as a subset of learning. The weak learner is built on this learning sets. The predictions are then obtained by combining the classifiers outputs by majority vote (Ting & Witten, 1997;Davison & Sardy, 2006). This method has shown its importance in recent work. Then, we propose to use this technique in this work to study the classifier ensembles based on formal concepts, since a limited number of studies have focused on the formal concepts in the context of parallel learning.
First, we present the basics of Formal Concept Analysis and several classification methods based on lattice concept or sub-lattice of concepts. Second, we present ensemble methods based on sequential and parallel learning and we propose a new method exploiting the advantages of the Dagging to generate and combine in a parallel way a weak concept learner. Then, we present classifiers based on FCA and an amelioration of this classifier. A comparative and experimental study is presented to evaluate the performance of concept ensembles based on certain criteria such as the number, variety, and type of classifiers. Finally, the comparative study shows the importance of parallel learning compared to sequential learning.
Let A ⊆  and B ⊆  be two finite sets. For both sets A and B, operators ϕ A ( ) and δ B ( ) are defined as (Ganter & Wille, 2005): Operator ϕ defines the properties shared by all elements of A. Operator δ defines instances which share the same properties included in set B. Operators ϕ and δ define a Galois connection between sets  and  (Ganter & Wille, 2005 Sets A and B are called, respectively, the extent (domain) and intent (co-domain) of the formal concept (Table 2). From the formal context O P R , , , we can extract all possible concepts organized as a complete lattice (called Galois lattice) (Ganter & Wille, 2005). We needed the following partial order relation ' ' is defined between two concepts, A B A B    (Ganter & Wille, 2005). Figure 1 represents the concept lattice (Galois lattice) of the context presented in Table 1.

Classification
The Galois lattice is a search space in which we evolve from a level to another, by validating the characteristics associated with the concepts (Meddouri & Maddouri, 2009). Classification based on FCA must determine the class of new objects by navigating on a lattice of formal concepts. Many classification systems exist in the literature using complete lattice or sub-lattice of concept (Fu et al., 2004).

Complete Lattice-Based Methods
Many classification systems use lattice concepts such as RULEARNER (Sahami, 1995), GALOIS (Carpineto & Romano, 1993) and NAVIGALA (Visani & al., 2011). The most common used   (Oosthuizen, 1988). From a pseudo-lattice, the classes of which contributes to the construction of this lattice as attributes, GRAND induces the most general rules. To do this, it selects the intermediate nodes of the immediate attribute node representing the class. To illustrate the classification approach based on a complete lattice of formal concepts, we apply GRAND on the initial formal context of Table 1. We obtained the concepts of Figure 2. From Figure 2, generating the classification rules are starting an attribute node that materializes 'Play=Yes' 2 . For example, GRAND select the concept number 12 (Figure 2). It looks for attribute nodes deriving this concept and gets attributes{ p p 6 7 , }. The following classification rule will be generated:

IF Temperature=Cool AND Humidity THEN Play=Yes
The instances labeled by the attribute-node 'Play=No' do not form a concept. According to (Oosthuizen, 1988) (Nguifo & Njiwoua, 2005, negatively labeled instances cannot induce a classification rule, only the positively labeled instances induce classification rule. The instances labeled by the attribute-node 'Play=No' do not induce a concept. Therefore, instances labeled negatively are considered instances associated to attribute-node 'Play=No'. The classification rules generated by GRAND from the formal context of Table 1 concern only the 'Play=Yes':   Table 1 5. IF Outlook=Overcast AND Temperature=Hot AND Windy THEN Play=Yes 6. IF Outlook=Sunny AND Temperature=Cool AND Humidity AND Windy THEN Play=Yes 7. IF Outlook=Rainy AND Temperature=Cool AND Humidity AND Windy THEN Play=Yes A critical overview of algorithms based on the extraction of formal concepts shows that existing algorithms in literature failed in their objectives. Indeed, almost all these algorithms have been focused on the extraction of all concepts of concept lattice, which increases the costs of extraction of classification rules and makes their use almost impossible for large databases. There are three common limits for systems based on the concept lattice. The complexity (temporally and spatially) of lattice generation is exponential. The navigation in huge search space is hard (Fu et al., 2004). And the used data is binary. For these reasons, many researchers focused on sub-lattice-based classification.

Sub-Lattice-Based Methods
The major difference between lattice and sub-lattice based classification is the number of concepts generated. However, their limit is the possible loss of information in a condensed data representation or a partial reproduction of the complete lattice. Systems like LEGAL (Liquiere & Nguifo, 1990), CIBLe (Njiwoua & Nguifo, 1999), CLNN&CLNB (Xie & al., 2002) and IPR (Maddouri, 2004) are characterized by their ability to build a part of the concept lattice and inducing classification rules.
In IPR for example, the construction of a concept cover is based on heuristic algorithms which reduces the complexity of learning. The concepts are extracted one by one. Each pertinent concept is given by a local optimization of Shannon entropy function. Each pertinent concept with associated majority class, constructs a rule. As an illustrative example, we applied IPR on the preceding formal context (Table 1), we obtained the concepts of Figure 3.
From the context of   One limit of IPR is the induction of redundant concepts (including same instances) (Maddouri, 2004) to cover all the formal context. For example, o 5 is covered by the concepts Several methods offer a partial reproduction of the concept lattice i.e., a generation of text classification rules, but unfortunately this leads to inefficient methods. These methods, which deal only with binary attributes, also incur some difficulties as the exponential complexity (in the worst case) and the generation of redundant concepts. We notice that with those methods based on sublattice classification, the constructed concepts are chosen based on inappropriate criteria (i.e., the depth of the lattice, the covering of the context, etc.) (Meddouri & Maddouri, 2009).

eNSeMBLe MeTHoDS
In machine learning, several researchers have focused on ensemble methods of classifiers.
These methods can improve the performance of individual and perceived 'weak' classifiers. Empirical studies have shown that the classifiers sets are often much more accurate than the individual classifiers (Bauer & Kohavi, 1999). Different theoretical explanations have been proposed showing the effectiveness of ensemble methods (Kleinberg, 2000).
Currently, there are two types of ensemble methods. The first type uses sequential learning to generate iteratively a set of classifiers, the same model from the same learning data which are weighted at each iteration. The second applies parallel learning of classifiers processing learning data independently. The predictions of the classifiers are then combined in an aggregation/vote or a stacking algorithm (Kleinberg, 2000).
In this paper, we present these two types of learning because of their importance in the literature and the lack of work that apply to the formal concepts, we propose two approaches. The first is based on the sequential learning i.e., that each classifier is built according to the performance of the previous classifier constructed. The second is based on the parallel learning.

Boosting
Boosting is an adaptive approach, which makes it possible to correctly classify an object that can be inappropriately classified by an ordinary classifier. The main idea of Boosting is to build many classifiers which complement each other, to build a more powerful classifier. At first, it selects a subset of instances from the learning data set (different subsets from the training data set in each iteration). Then, it builds a classifier using the selected instances. Next, it evaluates the classifier on the learning data set, and it starts again T times (T is the number of generated classifiers). Adaboost (Adaptive Boosting) is the most well-known method of Boosting for classifiers generation and combination.
For  class problem, let Y = { } 1,...,  be the class labels, with y Y i ∈ is the class label associated for each instance o i (i = 1 to n). To generate T classifiers in AdaBoost, the distribution of the weight of o i is initially determined as: On each iteration t} from 1 to T, we define: and we set: The error rate of h t is calculated on the weighted training set. If an instance o i is correctly classified by h t , then the weight of this instance is reduced, otherwise, is increased. The pseudo-loss of the classifier h t is defined as: The weights are then updated according to β t : The procedure is repeated T times and the result of AdaBoost is determined by the combination of the generated classifier outputs: The first variant of the AdaBoost algorithm is called Adaboost.M1 (Freund, 1995;Freundet al., 1998) that uses the previous process and stops it when the error rate of a classifier becomes over 0.5. The second variant is called AdaBoost.M2 (Freund & Shapire, 1996) which has the particularity of handling multi-class data and operating whatever the error rate is. In this study, we use AdaBoost.M2 since Adaboost.M1 has the limit to stop Boosting if the learning error exceeds 0.5.
According to (Warmuth et al., 2007), adaptive update of learning data in sequential learning (Boosting), increases the weight of those misclassified by the previous classifier and improves the performance of any learning algorithm (weak learner). However, the capacity of sequential learning has been challenged once highly noisy data are used. Noisy data will be ignored in parallel learning and possibly will spread equiprobable between Bootstraps or other subsets of training data resampled. Therefore, researchers have proposed techniques such as Bootstrapping (Breiman, 1996) and Disjoints Stratified (Ting & Witten, 1997) which consist of ignoring these noisy data or distributing on different sets of learning (Skurichina & Duin, 1998).
Additive Regression is an ensemble method that enhances the performance of a regression base classifier. Each iteration fits a model to the residuals left by the classifier on the previous iteration. Prediction is an accomplished by adding the predictions of each classifier. Reducing the shrinkage (learning rate) parameter helps prevent overfitting and has a smoothing effect but increases the learning time (Friedman, 1999).
DECORATE is an ensemble method for building diverse ensembles of classifiers by using specially constructed artificial training instances. Comprehensive experiments have demonstrated that this technique is consistently more accurate than the base classifier, Bagging and Random Forests. DECORATE also obtains higher accuracy than Boosting on small training sets and achieves comparable performance on larger training sets (Melville & Mooney, 2003).
MultiBoosting is an extension to the highly successful AdaBoost technique for forming decision committees. MultiBoosting can be viewed as combining AdaBoost with Wagging. It is able to harness both AdaBoost's high bias and variance reduction with Wagging's superior variance reduction. Using C4.5 as the base learning algorithm, MultiBoosting is demonstrated to produce decision committees with lower error than either AdaBoost or Wagging significantly more often than the reverse over a large representative cross-section of UCI data sets. It offers the further advantage over AdaBoost of suiting parallel execution (Webb, 1999).

Bagging
In parallel approach, Bagging is based on Bootstraps. The particularity of these training sets is to reduce the impact of hard instances to learn (called outliers and misleaders) (Skurichina & Duin, 1998). Each classifier is trained on a set of ′ n training instances ( ′ < n n ), drawn randomly with replacement from the original training set of size n. Such a training set is called a Bootstrap replicate of the original set. Each Bootstrap replicate contains, on average, 63.2% of the original training set, with many instances appearing several times. Predictions on the new instances are made by taking the majority vote of the ensemble. (Breiman, 1996) reports that majority vote can turn good classifiers into almost optimal classifiers. Bagging is typically applied to learning algorithms that are unstable i.e. a small change in the training set leads to a noticeable change in the model produced (Melville & Mooney, 2005). Since each ensemble member is not exposed to the same set of instances, they are different from each other. By voting the predictions of each of these classifiers, Bagging seeks to reduce the error due to variance of the base classifier. Bagging of stable learners, such as Naive Bayes, does not reduce error (Melville & Mooney, 2005). (Kuncheva et al., 2002) report that parallel learning improves the performance of unstable classifier as neural networks and decision trees. They report that Bagging is not beneficial to improve the performance of a linear classifier on large data. It will be then advantageous to use these methods with unstable classifier (such as CNC).
In the literature of data sampling methods, stratified sampling has proved efficiency (Ting & Witten, 1997). Disjoint and stratified data sets are more representative of the original training database. Learning from stratified data samples allows to generate more efficient classifier than those generated from the weighted data in the case of sequential learning classifiers. Dagging has the particularity to learn parallel from stratified data sets. We also propose to exploit this variant of parallel learning from stratified data to generate classifiers based on nominal concepts.

Input:
• T: the number of classifiers to generate.
• Learning data  of n instances: , ;

End
Dagging (Kotsianti & Kanellopoulos, 2007;Anyfantis et al., 2007) is described with more details in Algorithm 1. The learning algorithm is executed T times on various disjoint and stratified sets of learning instances. For each set, we do not obtain new instances, but we are satisfied to have a similar distribution to the initial sample of learning instances. The samples are obtained by randomly drawing ′ n instances without replacement in the training sample  , with ′ < n n . These samples respected the distribution of learning instances as classes.

PRoPoSeD MeTHoD BASeD oN FoRMAL CoNCePT ANALySIS
We have recently proposed a classifier based on FCA, which has the particularity to reduce training time and complexity. The principle is to find the most relevant concept for inducing a classification rule.

Classifier Based on Formal Concepts Analysis: CNC
The learning algorithm considers the whole of training instances and use nominal attributes (not only binary attributes). We note L, the number of nominal attributes AN with: The pertinent nominal concept within the data set is extracted by selecting the nominal attribute which minimizes the measure of Informational Gain (IG) calculated from the learning context: instance containing v j is redundant) and minimize IG of the corresponding attribute. This paralyzed effect does not bring a lot of diversity in the values of IG for the same attribute. In a diverse set of data, value of S v j ( ) is minimized and IG the corresponding attribute is more important. This phenomenon helps to better enhance the attributes of a diverse set. The stratified random sampling ensures the proportional presence of all the various subgroups within the data set. Clearly, finding the best attribute that maximizes the IG in a stratified set is more interesting than in another set. So, we are recommending learning CNC from stratified sets since it is based on the calculation of IG.
Once the nominal attribute is selected ( AN * ), we extract associated instances to each value v j from this attribute.
Proposition 1: From a nominal context (multi-valued), the $\delta$ operator is set by: Then, we look for the other attributes describing all the extracted instances (using the closure operator δ ϕ AN v j * = ( ) ). For this, we give the following proposition: Proposition 2: From a nominal context (multi-valued), the ϕ operator is set by: The condition part of the rule is made up by the conjunction of the attributes included in the intent: ) . The conclusion part of the rule is made up by the majority class. After that, it uses the discovered rule to classify the learning data set  and so our proposed learning algorithm of pertinent concept stops at this iteration.
Determine the majority class y * associated with δ AN v * = ( ) .

Induce the classification rule h CNC ;
Return h CNC . End

Parallel ensemble of CNC
The sequential learning builds sequentially a set of classifiers from the same model and from adaptively weighted data. Each classifier is generated based on the reweighting of training data which is dependent upon the performance of the previous classifier. It was found that the addition of classifiers could affect paralyzing in the sense that it does not lead to an implicit improving in the performance of sequential learning, but rather to its degradation because of overfitting (Kuncheva, 2005;Buhlmann & Hothorn, 2010). This is due to the repetitive sampling of the training data from similar distributions learning data. Looking for the best attribute for CNC is from similar training data will be the same closing of this attribute and the same as the previous classifier. It will not improve performance for this type of classifier in the case of sequential learning.
In parallel learning, data is first divided into random and independent subsets called Bootstraps. The same classification algorithm is then applied to each of these Bootstraps. Finally, the aggregation of generated classifiers is made by a simple majority vote or a weighted vote to predict the final class.
The best-known method, which is based on this type of learning is called Bagging (Bootstrap Aggregating) (Breiman, 1996;Breiman, 1996b). Methods using parallel learning are distinguished by the data sampling techniques they use to learn the classifiers from specific subsets. The peculiarity of learning from a Bootstrap is to combine the hard instances to the misleading instances in the training set (unlike the sequential approach). (Kuncheva et al., 2002) and (Breiman, 1996) report that parallel learning improves the performance of an unstable classifier such as Networks Neural or Decision Trees. However, this type of learning does not improve the performance of stable linear classifiers that generates. Kuncheva reported that learning parallel (Bagging) will not be beneficial to improve the performance of a linear classifier on large data. So, it will be advantageous to use learning parallel with a formal and unstable classifier (such as CNC).
To study the performance CNC, we selected 24 samples of different data (Presented with more details in next section. The performance of CNC is obtained according to the principle of 10 crossvalidation. In Table 3, we report the variance of the error rate for each data set.
These results show that CNC is an unstable classifier, considering that ensemble methods, which are based on the combination of several classifiers, are known to improve the performance of an individual week and unstable learner like CNC.

CNC From Disjoint and Stratified Learning Data
Recently, a great number of studies in machine learning has been concerned with parallel learning of classifiers that allow the improvement of single learner performances (Breiman, 1996). Parallel learning is known to improve the performance of any learning algorithm and discovering weak classifiers (weak learner). We propose to exploit the advantages of parallel ensembles methods to improve the performance of CNC.  (Bauer & Kohavi, 1999;Webb, 2000). Instead of applying resampling to obtain random Bootstraps (case of Bagging), Wagging uses random weights for learning instances. In (Bauer & Kohavi, 1999), gaussian noise is used to vary the weight. However, this can produce near zero weight, which leads to ignorance or disposal of instances to which these weights are associated.

Wagging (Weight aggregating) is another ensemble method considered as a variant of Bagging
We also report DECORATE (Diverse Ensemble Creation by Oppositional Relabeling of Artificial Training Examples) (Melville & Mooney, 2005;Melville & Mooney, 2004). This method generates a set of different classifiers with better performance. This set is generated iteratively. Each classifier is built individually and then added to the set. The first classifier is generated from the learning instances. Following classifiers are generated from this learning instances combined to others artificially generated. Labels of artificial instances are chosen such that they are different from the predictions of the initial learning instances. This provides more diversity in the learning data set. To maintain good performance of this ensemble, the error of each generated classifier is calculated. If this error is greater than the ensemble, generated classifier is rejected. An improvement is made to DECORATE to enable it learning from data whose attributes are multivalued (Kotsiantis, 2008). DECORATE is dedicated to numeric data rather than the nominal data. Therefore, it cannot be used for classifiers based on Formal Concept Analysis.
Dagging (Disjoint samples aggregating) is a variant of Bagging that creates a few disjoint groups with stratified data (Davison & Sardy, 2006;Ting & Witten, 1997), each considered as a learning subset. The predictions are obtained from a majority vote for supervised classification problems (Ting & Witten, 1997). This method has shown its importance in recent works. We propose to use it and to study the behavior of our CNC classifier based on stratified sampling and analysis the difference compared to random sampling. { } that will be combined by majority voting (Ting & Witten, 1997). In the case of learning from stratified samples, the characteristic of this type of sample is that it allows the learning algorithm to generate classifiers from representative samples having the same composition as the initial training set.

CoMPARATIVe STUDy
In this section, we compare the proposed method DNC (Dagging Nominal Concept) with existing ones based on Formal Concept Analysis: GRAND, IPR, CITREC and BNC. To compare the presented approaches, we consider their complexities. We also compare the DNC method with existing ones in literature: Bayes Net, Naive Bayes, SVM, 1-NN, Decision Stump, C4.5, Random Forest and Random Tree and we focus on their classification rates.

Comparison of Complexities
Concerning the complexities, the variable n means the number of instances and the variable m means the number of attributes. To calculate the complexity of DNC method, T is a variable which means the number of stratified data set. We examine the learning algorithm of pertinent concept and we estimate its complexity to O(nlog(n)+nm). The complexity of stratified sampling from a set is estimated to O(2n) (The complexity of listing all instances is O(n) and the complexity of the simple random sampling without replacement is estimated to O(n)). The complexity of closure operator is O(nm). Also, we examine the algorithm of Dagging and we estimate its complexity. The complexity of the Dagging iterations is O(T). The complexity of applying our weak classifier is O(2n+ nm).
To conclude, the complexity of the DNC method is estimated to O(T(2n + nm))=O(2n+ nm).

experimental Study
To compare the presented approaches, we implement them under Waikato Environment for Knowledge Analysis (WEKA) 3 (Hall et al., 2009) and we use the well-known data sets from UCI Machine Learning Repository (Asuncion & Newman, 2007). The chosen data sets were discretized with 2 discretional filters under WEKA. The first filter 4 used is an instance filter that converts a range of numeric and string attributes into nominal attributes. The second filter 5 used is an instance filter that converts a range of nominal attributes into binary attributes (to evaluate methods based on Formal Concept Analysis). These data sets are presented in Table 8 in 2 groups and depending on the data diversity to see if it affects the performance of Dagging. The 10 first data sets we will evaluate GRAND, RULEARNER, CITREC and IPR (the small data sets are reserved to these methods based on Formal Concept Analysis due to excessive consumption of memory resources). All other 24 data sets are used to evaluate some methods known in the literature as DNC. For each data set, we present respectively the number of instances, the number of numeric attributes (before discretization), the number of nominal attributes (after discretization), the number of classes and data diversity. The last column presents the ratio between the number of different vectors of instances (attributes) and the total number of vectors in each database (Table 5).
The performance of classifiers generated is evaluated in terms of error rates. To calculate these rates, the 10 Cross-validation method is used in WEKA the principle of which is to divide each base into 10 subsets. In turn, one subset is used for testing and the other ones for learning (Kohavi 1995).
In the next section, we will try to provide answers to the following questions: Does the classifier number influence the performance of Dagging of CNC? Is the Dagging of CNC more interesting than other classifiers? What is the best adaptive learning for CNC: sequential or parallel? What are the conditions under which CNC behaves better than other classifiers?

Influence of the Number of Classifiers on DNC
To study the performance of Dagging using CNC, we generated sets of 2 to 13 classifiers, and we reported their error rates in Table 9. Signs '-' indicates that the method cannot process the Weather data set (14 instances), due to the small number of instances not enough to construct sufficient disjoint and stratified subset of data. From this table, we report that the performance of odd sets of classifiers  is better than the pair sets. This is due to the voting rule of the combination that works best with an odd number of classifiers (Kunchevaet al. 2003). Dagging with more than 8 classifiers behaves better. The performance of these classifier ensembles is not correlated with the diversity of training data. We note that with 10, 11 and 13 classifiers, Dagging using CNC produces the best performance on 7 data sets from 24. The average error (for all the bases) occurred with 11 classifiers (14.45%) which is lower than that obtained with 10 classifiers (14.72%) or 13 classifiers (14.59%). That is why in the next experiments, we will retain this value for the number of classifiers (Table 6). Table 7 presents the performance of different methods based on Formal Concept Analysis including DNC. Signs '-' indicate that the method cannot process the sample data, due to excessive consumption of memory resources. As shown in Table 7, DNC has the specific ability to reduce the error rates compared to methods based on Formal Concept Analysis (GRAND, RULERNER, IPR and CITREC).  It is very important to determine the number of concepts obtained by each method based on Formal Concept Analysis. To determine the number of concepts in the Galois lattice, we use the GALICIA 6 software based on Godin algorithm which constructs the complete lattice of concepts (Godin et al. 1995;Valtchev et al. 2003). To determine the number of concepts generated by GRAND, RULERANER, CITREC, IPR and DNC, we have used the WEKA platform. We note that GRAND, RULERANER and IPR incur failures during their execution on Diabets, Segments and Sonar. GALICIA also reported a failure when run on the Sonar data.

Comparison with ACF Based Classifiers Classification Methods
As shown in Table 8, the concept lattice contains a great number of concepts. CITREC and DNC induces a small part of the lattice. DNC reached the best error rates with a small number of concepts. This shows that the concepts generated by DNC are more pertinent than those generated by IPR and CITREC. DNC gives a small number of concepts in the least possible time compared to the other approaches. Table 9 presents the performance of different classification methods from the literature (Bayes Net, Naive Bayes, SVM 7 , IB1 8 , Decision Stump, C4.5 9 , Random Forest and Random Tree) on numeric data. We note that all this classification methods incur failures during their execution Japanese vowels since it cannot handle nominal class attribute. Table 10 presents the performance of different classification methods cited previously, compared to our proposed method DNC on discretized data. We report that the used discretization filter data reduce the error rates of Naive Bayes and SVM. As shown in this table, DNC has the specific ability to reduce the error rates together with the best performance for 14 discretized data sets (Contact lenses, Weather, Lymphography, Sonar, Diabets, Iris, Balance scale, Waveform, German credit, Japanese vowels, CMC, Solar flare, Page blocks and Yeast). DNC has the best performance for all the data sets (average of 14.45%), then SVM (average of 16.29%) compared to Random Forest (average of 17.37%). (Meddouri & Maddouri, 2010) found that the sequential learning is beneficial for classifiers having decision tree structure such as C4.5 and Id3. However, the CNC is among the worst classifiers. Our objective here is to study the behavior of the classifiers in parallel learning. The error rates of the ensembles of generated classifiers are reported in Table 11. These results show that CNC holds the best performance for 17 data sets from the 24 (average of 14.46%). SVM is better than Random Forest, Bayes Net and Random Tree. Decision Stump produced higher error rates than the rest of the classifiers (average of 34.84%).

Influence of the Classifier Type in Dagging
For correlated data sets such as Pages blocks and Solar Flare (with diversity of 23.14% and 34.3%, respectively) the error rates of CNC are lower. Since the data diversity of Yeast is 22.34%, the error rates are quite higher. In general, the diversity of data is not correlated with the performance of nominal classifiers.
In conclusion, we can note from these experiments that parallel learning is interesting for CNC. The other classifiers are rather used in sequential learning as shown in (Meddouri & Maddouri, 2010).

Comparison of ensemble Methods
According to the literature, the relationship between classifiers and the ensemble methods is not clear: for example, we do not know whether it is better to use Boosting, Bagging or Dagging for a given classification problem. Further work using the Bagging without a priori knowledge on diversity in ensembles of classifiers that generates (Breiman, 1996). In (Ting & Witten, 1997), the authors have shown, theoretically and experimentally, the importance and the reliability of Dagging. (Meddouri et al. 2012) noticed that CNC is not good enough with sequential learning on data sets of different sizes. For this, we used 24 samples of data to generate 11 sets of nominal classifiers such as CNC in Boosting, Bagging and Dagging. Table 12 presents the results of all these methods, in terms of error rate and training time. We can see that Dagging is the best method producing low error rates for all generated ensembles.
All the error rates of Boosting are higher than those of Dagging. In addition, Boosting is 6 times slower than Dagging. We also report that Dagging is 2 times faster than Bagging.
In our experiments, the use of 11 sets of nominal classifiers based on Formal Concept Analysis, in Dagging, is more interesting than by Bagging and Boosting. The parallel learning of such classifiers is far better than the sequential learning.

CoNCLUSIoN
Formal Concept Analysis is an interesting formalism to study machine learning and classification methods. It allows a full construction of the concepts and the dependence relationships between concepts to build a lattice of Formal Concepts. We report common limits among all the supervised learning methods based on Formal Concept Analysis: absence of the adaptive aspect and the generation of concepts is either exhaustive or non-contextual.
In this paper, we propose a new classifier based on nominal concept and we showed that it is better than the other methods based Formal Concept Analysis. We select the nominal attribute which maximizes the Informational Gain from the nominal data. Pertinent concepts are calculated from closure operators associated to this nominal attribute. A classification rule is obtained by associating a majority class to the extension of the pertinent concept.
Since it is a weak classifier, we propose to improve its performance by using ensemble methods. We have recently proposed a variant of Dagging to generate classifiers based on Formal Concept. Recent work has encouraged its use for linear classifiers. No studies have focused on the generation of classifiers by formal concept type Dagging. We recommend a parallel learning by Dagging for classifiers such Formal Concept and the sequential learning for the other classifier types. In parallel learning, a few classifiers are sufficient for obtaining better performance than the use of individual one. Data diversity can affect the whole.
More experiments are possible on larger data sets and with other ensemble methods such as Random Forests. Many improvements on the ensemble methods can be brought. DNC methods used majority vote for classifier combination. A variety of voting rules already exist. A study of these rules can be beneficial to improve the performance of CNC ensembles.