{'fraction_non_alphanumeric': 0.033125805456228556, 'fraction_numerical': 0.011181093786869475, 'mean_word_length': 4.989780387040661, 'pattern_counts': {'":': 1, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Metafeatures-based Rule-Extraction for Classifiers on Behavioral and Textual Data

Machine learning models on behavioral and textual data can result in highly accurate prediction models, but are often very difficult to interpret. Rule-extraction techniques have been proposed to combine the desired predictive accuracy of complex"black-box"models with global explainability. However, rule-extraction in the context of high-dimensional, sparse data, where many features are relevant to the predictions, can be challenging, as replacing the black-box model by many rules leaves the user again with an incomprehensible explanation. To address this problem, we develop and test a rule-extraction methodology based on higher-level, less-sparse metafeatures. A key finding of our analysis is that metafeatures-based explanations are better at mimicking the behavior of the black-box prediction model, as measured by the fidelity of explanations.


Introduction
Technological advances have allowed storage and analysis of large amounts of data and have given industry and government the opportunity to gain insights from thousands of digital records collected about individuals each day [59]. These "big behavioral data"-characterized by large volume, variety, velocity and veracity, and defined as data that capture human behavior through the actions and interactions of people [75]-have led to predictive modeling applications in areas such as fraud detection [84], financial credit scoring [53,20,80], marketing [87,59,11] and political science [68]. Sources of behavioral data include, but are not limited to, transaction records, search query data, web browsing histories, social media profiles, online reviews, and smartphone sensor data (e.g., GPS location data). Textual data are also increasingly available and used. Example text-based applications are automatic identification of spam emails [4], objectionable web content detection [56] and legal document classification [13], just to name a few.
Behavioral and textual data are very high-dimensional compared to traditional data, which are primarily structured in a numeric format and are relatively low-dimensional [62,59,19]. Consider the following example to illustrate these characteristics: The prediction of personality traits of users based on the Facebook pages they have "liked" [45,59]. A user is represented by a binary feature for each unique Facebook page that exists, with a 1 if that page was liked by the user and 0 otherwise, which results in an enormous feature space. However, each user only liked a relatively It is important to note that the high-dimensional, sparse nature of behavioral and textual data alone does not necessarily lead to complex prediction models. If many behavioral or textual features are irrelevant for the prediction task, applying dimensionality reduction or feature input selection prior to modeling, or using strong model regularization can result in models having high predictive performance, while still being interpretable. However, previous research shows that all of these techniques result in worse predictive performance compared to models that exploit the full set of behavioral or textual features for making predictions [39,41,15,57,19,21]. By means of a learning curve analysis on a benchmark of 41 behavioral data sets, [19] demonstrate that, when mining text or behavior, many features contribute to the predictions. Similar results have been found by [15] and [41] for behavioral data, and by [39] for the analysis of textual data. In other words, the dimensionality and sparsity of the data combined with many relevant features drive the "black-box" nature of any model trained on behavioral and textual data. We represent a classification model trained on behavioral or textual data X F G as C BB , where BB stands for "black-box".
Explainability has emerged recently as a key business and regulatory challenge for machine learning adoption. The relevance of global interpretability of classification models is well-argued in the literature [2,23,53,40]. 2 In the process of extracting knowledge from data, the predictive performance of classification models alone is not sufficient as human users need to understand the models to trust, accept and improve them [83]. Both the United States and the European Union are currently pushing towards a regulatory framework for trustworthy Articial Intelligence, and global organizations such as the OECD and the G20 aim for a human-centric approach [25]. In high-stake application domains, explanations are often legally required. In the credit scoring domain, for example, legislation such as the Equal Credit Opportunity Act in US Federal Law [82] prohibits creditors from discrimination and requires reasons for rejected loan applications. Also in lower-stakes applications, such as (psychologically) targeted advertising [59,62] or churn prediction [86], explanations are managerially relevant. Global interpretability allows to verify the knowledge that is encoded in the underlying models [2,36]. Models trained on big data may learn incorrect trends, overfit the data or perpetuate social biases [11]. Furthermore, explanations might give users more control of their virtual footprint. [58] argue that insight into what data is being collected and the inferences that can be drawn from it, allow users to make more informed privacy decisions [58]. Moreover, global model explainability can help to induce new insights or generate hypotheses [2,75].
Rule-extraction has been proposed in the literature to generate global explanations by distilling a comprehensible set of rules (hereafter referred to as "explanation rules") from complex classifiers C BB . The complexity of the set of rules is largely restricted so that the resulting explanation is understandable to humans. Rule-extraction in the context of big behavioral and textual data can be challenging, and, to our knowledge, has thus far received scant attention. Because of 2 Explanations for model predictions vary in scope: A method either generates global or instance-level explanations [56,71]. We focus on global explanations that give insight in the model's behavior over all possible feature values and for all instances. Instance-level explanations, on the other hand, explain a single model decision. For example, when mining behavior or text, an Evidence Counterfactual is an instance-level explanation that shows a minimal set of features such that, when changing their feature values to zero, the predicted class changes [56,71]. the data characteristics, rule-extraction might fail in their primary task (providing insight in the black-box model) as the complex model needs to be replaced by a set of hundreds or even thousands of explanation rules [36,78].
This article addresses the challenge of using rule-extraction to globally explain classification models on behavioral and textual data. Instead of focusing on rule-extraction techniques themselves 3 , this article leverages an alternative higher-level feature representation X M F ⊂ R n×k , where MF stands for "metafeatures" and k represents the number of metafeatures. Metafeatures are expected to improve the fidelity (approximation of the black-box classification model), explanation stability (same explanations for slightly different training sessions -a concept we introduce, which we will be calling just stability) and accuracy (correct predictions of the original instances) of the extracted explanation rules. For simplicity, we only focus on classification problems. Our main claim is that metafeatures are more appropriate, in specific ways we discuss, for extracting explanation rules than the original behavioral and textual features that are used to train the model. This article's main contributions are threefold: (1) We propose a novel methodology for rule-extraction by exploring how higher-level feature representations (metafeatures) can be used to increase global understanding of classification models trained on fine-grained behavioral or textual data; (2) we define a set of quantitative criteria to assess the quality of explanation rules in terms of fidelity, stability, and accuracy; and empirically study the trade-offs between these; and lastly, (3) we perform an in-depth empirical evaluation of the quality of explanations with metafeatures using nine data sets, and benchmark their performance against the explanation rules extracted with the fine-grained features. We aim to answer the following empirical questions: • How do explanation rules extracted with metafeatures compare against rules extracted with the fine-grained behavioral and textual features across different evaluation criteria (fidelity, stability, accuracy)? • How does the fidelity 4 of explanation rules vary over different complexity settings? • To what extent do the fidelity and stability of explanation rules extracted with metafeatures depend on a key parameter of our metafeatures-based rule-extraction methodology, that is the parameter k that represents the number of metafeatures?
2 Related work

Rule-extraction
In the Explainable Artificial Intelligence (XAI) literature, rule-extraction falls within the class of post-hoc explanation methods that use "surrogate models" to gain understanding of the learned relationships captured by the trained model [53,63]. The idea of surrogate modeling is to train a comprehensible surrogate model (the white-box C W B ) to mimic the predictions of a more complex, underlying black-box model 5 C BB [23]. We define a black-box model as a complex model from which it is not straightforward for a human interpreter to understand how predictions are made. In this article, we consider any classification model (linear, rule-based or nonlinear) utilising a large number of features as a black-box (because of the specic nature of the data described in the Introduction, a large number of features are typically used in the final models); which is different from previous research that only considers highly-nonlinear models as black-boxes [2,53,23,54].
In the machine learning literature, small decision trees and rule-based models with few rules have been argued to yield the most comprehensible classification models [83,28], making them good candidates to use as white-box models C W B to extract a set of explanation rules (known as "rule-extraction"). 6 It is important to note that the complexity of the rules needs to be restricted so that the resulting explanation is comprehensible to humans [53,54].
Rule-extraction can be used for two purposes. First and foremost, one may be interested in knowing the rationale behind decisions made by a classification model C BB and verify whether the results make sense in practice. The goal is to extract comprehensible rules that closely mimic the black-box, that is measured by what is called "fidelity". Alternatively, the goal can be to improve the "accuracy", namely, the generalization performance of a white-box model 3 In this article, we use the decision tree algorithm CART as the rule-extraction algorithm. We leave the comparison of different rule-extraction techniques (Ripper [16], C4.5, and so on) as well as more advanced variants like active learning-based rule-extraction [40,17] to future research. Our main focus is on empirically assessing the value of using metafeatures for extracting explanation rules. 4 As the main goal of rule-extraction is to mimic the behavior of black-box models with a set of rules, we focus on fidelity instead of, say, accuracy, as discussed in Section 4.4. Our methodology and analysis can be adapted to also study the accuracy of explanations. 5 We will interchangeably refer to this model as the black-box model or the underlying model that we want to interpret globally. 6 Note that, in the literature, also linear models with a small number of features have been proposed as surrogate models to approximate a prediction model's behavior [72]. In this article, we focus on rule-based models as surrogates.
(e.g., a small decision tree or a concise set of rules) by approximating the black-box [54,36]. In this article, we discuss most results in terms of fidelity instead of accuracy as our focus is on developing global explanations that "best mimic the black-box" -but all our analyses can also be done using accuracy as the main metric.
Rule-extraction methods use the mapping of the data to the predicted labels, i.e., the input-output mapping defined by the model C BB [2,53,36]. The idea behind this approach is that the similarity between the black-box and white-box model (the fidelity) can be substantially improved by presenting the labels predicted by the black-box modelŷ={ŷ i } n i=1 to the white-box model, instead of the ground-truth labels y={y i } n i=1 [54,40].

Challenges of rule-extraction for high-dimensional data
The vast majority of the rule-extraction literature has focused on improving the fidelity and scalability of rule-learning algorithms. However, despite some very impressive and promising work [2,53,54,23,40], the rule-extraction techniques are mostly validated on low-dimensional, dense data, such as the widely-used set of benchmark data from the UCI Machine Learning repository [5]. These data have feature dimensions going up to 50 features. We identify at least three challenges in regard to rule-extraction to explain classifiers on fine-grained behavioral and textual data: 1. Complexity of the explanation rules. In the context of high-dimensional data with many relevant features, rule-extraction might fail to provide insight in the black-box model as the black-box model needs to be replaced by a large set of rules [53,54,36]. [78] apply rule-extraction on (real-world) textual data and show that rule learners can closely approximate the underlying model, but at the cost of being very complex (hundreds of rules). A related challenge that stems from rule-based learners not being very adept at handling high dimensionality, is their high-variance profile that can result in overfitting [46,19]. 2. Computational complexity. It is not straightforward for every existing rule-learning algorithm to be used for high-dimensional data, because the learning task can become computationally too demanding [2,78]. Some algorithms, such as Ripper [16], are not able to computationally deal with problem instances having large-scale feature spaces [78]. 3. Fine-grained feature comprehensibility. [23] questions the usefulness of rules extracted from models trained on behavioral or textual data. For example, when rules are learned from a model initially trained on a "bag-ofwords" representation of text documents, the antecedents in a rule include individual words out of context. This can reduce the semantic comprehensibility of the explanations. Likewise, for digital trace data, we can question the comprehensibility of a single action (e.g., a single credit card transaction, a single Facebook "like") taken out of its context, that is, the collection of all behaviors of an individual.
Because of the above challenges, it is questionable whether fine-grained behavioral and textual features are the best representation for extracting global explanation rules to achieve the best explanation quality in terms of fidelity, stability, and accuracy. This motivates our approach to use a metafeatures representation instead. It is not clear a priori whether such a representation will improve the quality of explanations of models on behavioral and textual data, making this a key empirical question that we study in this article.

Motivation
As previously introduced, behavioral and textual data suffer from high dimensionality and sparsity. For this reason, the features individually may exhibit little discriminatory power to explain the black-box model. Because of the low coverage that characterizes such sparse features, a single feature is not expected to "explain" much of the classifications of the underlying model. The feature will only be active (nonzero) for a small fraction of all data instances, and therefore, the coverage of an explanation rule with a single behavioral or textual feature is likely to be low [77,56,12,78]. 7 We address the data sparsity by mapping the fine-grained, sparse features X F G ⊂ R n×m onto a higher-level, less-sparse feature representation X M F ⊂ R n×k (to which we refer as "metafeatures"), where m and k respectively represent the dimensionality of the original features and metafeatures. Existing research has experimented with the idea of using higher-level features other than the actual features used by the model to extract explanations [72,12,43,50]. In the field of image recognition, for example, the input pixels are not straightforward to interpret, hence researchers have proposed to use a patch of similar pixels (super-pixels) for generating explanations of image classifications [72,90]. Another example stems from the field of natural language processing, where [12] cope with data sparsity by clustering similar features by their frequency in large data sets. All of these approaches have, however, not been used before in the context of rule-extraction for models on big behavioral and textual data.

Desired properties
We propose the following set of properties for engineering metafeatures: 1. Low dimensionality. We want the dimensionality k of the metafeatures to be smaller than the dimensionality m of the original feature space: k<<m. A lower feature dimension may lead to more stable explanation rules [1]. Moreover, the computational burden for extracting rules with metafeatures is likely to be much lower compared to rule-extraction with high-dimensional data [2,78]. 2. High density. This property relates to the coverage of a metafeature, which we want to be higher compared to the coverage of fine-grained features [12]. In other words, there should be more instances for which a metafeature is active (nonzero) compared to the fine-grained features. The higher density (lower sparsity) of the metafeatures is expected to increase the fidelity and accuracy of explanation rules resulting from the higher coverage of rules predicting the non-default target class (often the "class of interest"). 3. Faithful to the original feature representation. This property is in line with prior research suggesting that the representation of the original data instances in terms of metafeatures should preserve relevant information to discriminate between the predicted labelsŷ [1]. The metafeatures should preserve the predictive information of the original features as the black-box model is trained on the latter. It is important that the extracted rules using metafeatures can reach a high level of discriminatory power in regard to the true predictions being made, because this will result in a better approximation of the underlying model as measured by fidelity. In the experiments, we use the test fidelity of the explanations as a proxy to measure the faithfulness of the metafeatures X M F to the original features X F G . In addition, we use the Gini index to measure the predictive information of each metafeature. 8 4. Semantic comprehensibility. Metafeatures should have a human-comprehensible interpretation [1]. For example, Facebook "likes" can be grouped into semantically-meaningful categories (e.g., "Machine Learning") and GPS location data can be categorized into venue types (e.g., "Concert halls"). This property is subjective in nature and depends on the application domain and the expectations of users who interact with the model (explanations) [91,9,36,37]. We do not explicitly measure the comprehensibility of explanations with (meta)features, as this would require experimentation with people, an important research direction to explore if indeed metafeatures improve the quality of explanations in the other dimensions we study here (fidelity, stability, accuracy). In this article, we make the assumption that the resulting metafeatures are semantically meaningful. In Section ??, we demonstrate how metafeatures generated with a data-driven method (e.g., Nonnegative Matrix Factorization) can be interpreted, based on common practices described in the literature [88,64,67,47,21]. Note that the metafeatures that are manually crafted (the "domain-based" metafeatures described in Section 4.2) are, by design, comprehensible to humans.

Metafeatures-based Explanation Rules
We introduce and validate a methodology to extract explanation rules from a complex model C BB trained on behavioral and textual data X F G . The steps of the proposed methodology are summarized in Fig. 1 and discussed below.

Model building and predicting labels
From the behavioral and textual data X F G (having n instances and m features) we train and test the black-box model C BB . The model is trained on a subset of α×β×n instances (the training set X F G,train with corresponding labels y train ) and hyperparameters are optimized using a holdout set of α×(1 − β)×n instances (the validation set X F G,val with labels y val ). Finally, the generalization performance of the black-box model is evaluated on an unseen part of the data (the test set X F G,test with labels y test ) that contains (1 − α)×n instances. 9 The trained black-box model is used to make predictionsŷ for all instances in the data set (training, validation and test data), which will thereafter be used to train, fine tune and test our white-box model C W B (the explanation rules).

Generating metafeatures X M F
We need to specify a feature transformation process to group behavioral and textual features X F G with similar properties together in metafeatures X M F [12]. There are various approaches for generating metafeatures from the original features, either by manually crafting them using domain knowledge [63] or by automatically obtaining them by means of datadriven feature engineering techniques, such as (un)supervised dimensionality reduction. In the following, we use DomainMF and DDMF as abbreviations for domain-based metafeatures and data-driven metafeatures respectively.
Domain-based metafeatures. One way of generating metafeatures from the original features is to group features together in domain-based categories that are manually crafted by experts [63,1]. For example, for the Facebook "like" data, individual Facebook pages can be grouped together in predetermined categories, for example, pages related to "Machine Learning". This human-selected set of metafeatures can then be used to extract simple rules to explain model predictions, which represent the relative importance of these domain-based metafeatures in the prediction model. In this article, we mathematically denote the domain-based metafeatures as X DomainM F ⊂ R n×k .
Data-driven metafeatures. Alternatively, metafeatures can be generated via a data-driven approach, such as matrix factorization-based dimensionality reduction. 10 The idea is to increase density by representing the data in a lower dimensional space without too much loss of information. The original data matrix X F G with n unique instances and m unique features is split into two matrices L n×k and R k×m such that: X F G ≈LR. The k columns of L are the metafeatures, and each instance will have a representation in the new k-dimensional space. The matrix R, represents the relationship between the new metafeatures and the original features [15].
Metafeatures group together related features. The quality of the metafeatures depends on the number of extracted metafeatures k: A value of k that is set too high results in many highly-similar categories, whereas a low value of k tends to generate overly-broad metafeatures. The intended goal of generating metafeatures in this article is to use them for rule-extraction, and consequently, we optimize the number of k such that the out-of-sample fidelity of the rules is maximal (we use a validation set to fine tune the value of k). We consider values of k from 10 up to 1000 [15]. Note that we should not be concerned with generating too many metafeatures because we only need to interpret the ones that are part of the final explanation rules (this is demonstrated in Section 4.5).
For generating metafeatures based on matrix-factorization-based dimensionality reduction, we first approximate the original training data X F G by two matrices L and R for a given number of metafeatures k (step 1 in Fig. 6 in Appendix 1). Matrix R maps each metafeature to the original fine-grained features. We ensure mutual exclusivity by transforming matrix R into a binary matrix R binary , where 1 represents the maximum element for each column (fine-grained feature) of R and all other elements are 0 (step 2 in Fig. 6). In other words, each feature only belongs to one metafeature. Next, we map the original matrix X F G to X by multiplying X F G with the transposed binary matrix R T binary (step 3 in Fig. 6). Finally, matrix X is normalized over the number of active (fine-grained) features per instance (e.g., total number of behaviors or words) to become matrix X DDM F −k that represents the metafeatures per instance (step 4 in Fig. 6). We found that the normalized matrix X DDM F −k produced better results (as measured by test fidelity of the explanations) than utilizing the original matrix X or even a binary matrix derived from X . We apply the binarization of matrix R to make the explanation rules more interpretable and semantically meaningful.
In this article, we will experiment with two well-established dimensionality reduction methods based on matrix factorization: Non-negative Matrix Factorization (NMF) and Singular Value Decomposition (SVD). NMF is applied in multiple domains to decompose a non-negative matrix into two non-negative matrices [49]. In most real-life applications, negative components or subtractive combinations in the representation are physically meaningless. Incorporating the non-negativity constraint thus facilitates the interpretation of the extracted metafeatures in terms of the original data [89,47,15]. SVD is a popular technique for matrix factorization across a wide variety of domains such as text classification [34] and image recognition [35]. SVD is computed by optimizing a convex objective function and the solution is equivalent to the eigenvectors of the data matrix. We will implement these dimensionality reduction techniques using Python's Scikit-learn package [66].
An important assumption we make is that the resulting data-driven metafeatures are semantically meaningful. While the obtained metafeatures are not always guaranteed to be interpretable, especially NMF has been shown to provide interpretable results for fine-grained data applications [67,48], as compared to other techniques like SVD. Usually, metafeatures are interpreted by looking at the top-weighted features [88,64,67,47,21]. It is important to note that only the metafeatures that are part of the final explanation rules need to be interpreted. 11

Extracting explanation rules
Both rule and decision tree learning algorithms can be used for rule-extraction. Since trees can be converted into rules, we also use tree algorithms for rule-extraction [53,37,55]. A full review of these techniques is beyond the scope of this article, but we will shortly describe CART 12 , as this is the technique used in our experiments.
CART can be used for both classification and regression problems and it uses the Gini index as a splitting criterion, which measures the impurity of nodes. The best split is the one that reduces the impurity the most. We apply CART to the data where the target variable is changed to the black-box predicted class labelŷ instead of the ground-truth labels y (see Section 2.1).
The number of extracted explanation rules can be used as a proxy for human comprehensibility. 13 Restricting the complexity of the rule set is also motivated by research on how people make decisions: based on relatively simple rules to avoid excess cognitive effort [29,31] due to cognitive limitations [79]. In the context of consumer decisionmaking for example, [31] argue that decision rules should incorporate "cognitive simplicity": Rule sets should consist of a limited number of rules, each with a small number of antecedents. Finally, it is important to note that the concept of comprehensibility in the context of explanation rules comprises many different aspects, such as the size of the explanation, but also the specific application context and subjective opinion and expectations of the end user, which makes it difficult to measure comprehensbility in a generic way [37,9,91]. In line with cognitive simplicity arguments [31,79], in the experiments in Section 6, we restrict the complexity of the explanations to at most 32 rules each consisting of at most five antecedents (this is equivalent to a tree depth of at most five). 11 We will demonstrate in Section 4.5 how to interpret metafeatures that are part of the explanation by looking at the top-weighted features per metafeature. 12 CART is readily available from the Scikit-learn library in Python. 13 A general assumption in the literature is that linear models with few parameters or rule-based models with few rules are more comprehensible than linear models with many parameters or rule sets with many different rules [28,31,37].

Evaluating explanation rules
Fidelity. First and foremost, the explanation rules are evaluated on how well they approximate the classification behavior of the underlying model. Fidelity measures the ability of the rules to mimic the model's classification behavior from which they are extracted. Let {x i ,y i } n i=1 represent the labeled data instances and y WB andŷ respectively the white-box and black-box predicted labels. Fidelity is expressed as the fraction of instances for which the label predicted by the explanation rules (the white-box predicted label) equals the black-box predicted label [17,36]: While most of our analysis is using fidelity, we can extend the fidelity to "f-score fidelity" (f-fidel), which may be a more appropriate evaluation metric when there is a (predicted) class imbalance. The f-fidel is defined as the harmonic mean between precision and recall (w.r.t. the predicted labelsŷ rather than the true labels y). More precisely, the formula of f-fidel is 2·precision·recall precision+recall , where the precision of the classifier is the fraction of positively-predicted instances that is correctly classified and the recall refers to the fraction of positive instances that is correctly classified as a positive.

Stability.
A second important factor is explanation stability -which we will call stability from here on. Users, businesses, or regulators may have a hard time accepting explanations that are unstable (meaning small changes in the data lead to large change in explanations of the black-box model), even if the explanation has shown to have high fidelity and comprehensibility [83]. [81] distinguishes two types of stability: syntactic and semantic stability. Semantic stability is often measured by estimating the probability that two models learned on different training sets, will give the same prediction to an instance. On the other hand, syntactic stability measures how similar two explanations are (e.g., the overlap of features in two different explanations), and is more specific to a particular explanation representation [81]. We argue that syntactic stability is the most relevant type of stability in the context of explaining classification models. To the best of our knowledge, it remains an open question how to measure syntactic stability for different explanation representations, such as rules and trees. We propose a procedure based on the Jaccard coefficient to measure syntactic stability of explanation rules. More specifically, by measuring the overlap of features that are part of the explanations extracted from slightly different subsets of training data. 14 To compute the stability of explanation rules extracted using different data representations ∀X∈{X F G , X DDM F −k , X DomainM F } and the black-box predicted labelsŷ, we propose the following procedure: Prior research has shown that explanations that rely on high-dimensional data tend to be less robust compared to methods that operate on higher-level features [1]. For this reason, we expect that the extracted explanation rules with metafeatures will be more stable over different training sessions compared to the rules with the original behavioral and textual features. It is important to note, however, that for the metafeatures generated with a data-driven approach (X DDM F −k ), the computed stability of the explanations also depends on the metafeature generation method (e.g, 14 The procedure is based on the work of [27] who compare sets of patterns using the Jaccard coefficient. 15 In the experiments, we set the number of bootstrap samples B to 10. 16 How many top-features to compare between metafeatures and the cut-off value c, are parameter values that need to be set in advance. We consider looking at the top-20 features in a metafeature and a cut-off value of c=0.5 as suitable choices based on the literature on interpreting factors obtained from dimensionality reduction [21]. Accuracy. Rule-extraction has also been used to increase the generalization performance of white-box models C W B , as measured by accuracy. [54] show that rules that mimic the behavior of an underlying, better-performing model can become more accurate compared to the rules learned from the original data and the corresponding ground-truth labels y. Accuracy is defined as the fraction of correctly classified instances by the explanation rules [36]:

Experimental setup
The experiments in this article explore the performance of explanation rules with metafeatures versus the original features on which the model is trained. We make a distinction between domain-based metafeatures (X DomainM F ) and metafeatures generated with a data-driven method (X DDM F −k ). The dimensionality reduction parameter k determines the number of metafeatures. The parameter k is fixed for the domain-based, but a hyperparameter for the data-driven metafeatures. We evaluate the performance on a suite of classification tasks using nine behavioral and textual data sets. Fig. 7 in Appendix 5 summarizes the experimental procedure for evaluating the fidelity, f-fidel and accuracy of explanations using five-fold CV, and the explanation stability using bootstrapping.

Data sets and Models
Our experimental data comprise seven behavioral and two textual data sets. The data sets are summarized in Table 1.
The Movielens100 and Movielens1m [30] data sets contain movie viewing data from the MovieLens website. We focus on the task of predicting the gender of these users. The Airline data 17 contains Twitter data about American airlines, and the task is to predict (positive) sentiment. The Facebook "like" data collected by [68] (Facebook) contains likes from over 6,000 individuals in Flanders (Belgium) and is used to predict gender. The Yahoomovies data 18 also contains movie viewing behavior, from which we predict gender. The Tafeng data [33] consists of fine-grained supermarket transactions, where we predict the age of customers (younger or older than 30) from the products they have purchased. The 20news data [92] contains about 20,000 news posts. For this data, the task is to predict whether a post belongs to the topic "atheism", based on the words of the post. Another behavioral data set is the Libimseti data [8], which contains data about profile ratings from users of the Czech social network Libimseti.cz. The prediction task is, again, the gender of the users. Lastly, the Flickr data [10] contains millions of Flickr pictures and the target variable is the popularity of a picture (the number of comments it has). We train Logistic Regression models with l2-regularization ( 2 -LR) 19 with the Scikit-learn library (Python). For training the classification models, we use 80% of the data, and the remaining 20% of the data is used for testing the models. For fine tuning hyperparameters of the model, we use a validation set (20% of the training data). More specifically, the regularization parameter C of the 2 -LR model is selected based on the accuracy on the validation set. For preprocessing the text data, we remove stopwords and lemmatize tokens using the NLTK package in Python, and then use tf-idf 20 vectorization [39,56].
Measuring accuracy in practice requires discrete class label predictions, which we obtain by comparing the predicted probabilities to a threshold value t and assigning instances with a predicted probability that exceeds this threshold a positive predicted label. In practice, the choice of the threshold value t depends on the costs associated with false positives and false negatives. In this article, the exact misclassification cost are unknown, and for this reason we compute the threshold value t such that the fraction of instances that are classified as positive equals the fraction of positives in the training data [51]. Table 2 indicates the generalization performance of all models for each data set over five folds. In addition to the accuracy, we also report the f-score, precision and recall.
To extract explanation rules with the CART algorithm, we use the DecisionTree model of the Scikit-learn library. For controlling the complexity of the extracted rules, or equivalently the depth of the tree, we change the value of the max_depth parameter. We let the depth of the tree vary from 1 to 5 such that the explanations are cognitively simple (which we motivated in Section 4.3).
We extract explanation rules with the original features X F G (on which the classification models are trained) and the metafeatures X M F , and the predicted black-box labelsŷ. In the experiments, we generate data-driven metafeatures X DDM F −k based on two approaches (see Section 4.2): NMF and SVD. In the experimental results, we mainly discuss the explanations with DDMF generated via NMF (simply denoted by DDMF), that showed the best (fidelity) results. For the Facebook and Movielens1m data, we also extract explanations with domain-based metafeatures. 19 In the literature, Logistic Regression with 2-regularization has shown to be the best-performing classification model for behavioral and textual data [19]. 20 Tf-idf is short for term frequency and inverse document frequency. Table 3: Evaluation of explanation rules for 2-LR model using fine-grained features (FG) and data-driven metafeatures (DDMF) with optimal dimensionality reduction parameter k in parentheses. The best performance values (FG vs DDMF) are indicated in bold. The average fidelity, f-fidel and accuracy on the test data are reported over five-fold CV. The stability is measured over 10 bootstrap samples. The optimal complexity (tree depth) is shown in the last column. We also report the results for explanations with DDMF generated via SVD (DDMF-SVD). For Facebook and Movielens1m, we also report results for the domain-based metafeatures (DomainMF).

Experimental results
We compare explanation rules for black-box models extracted with metafeatures against those extracted with finegrained features, across different classification tasks, data sets and evaluation criteria. As mentioned, our main goal is to better understand how metafeatures affect these different criteria and their trade-offs.
6.1 Are metafeatures better than the original features for explaining models on behavioral and textual data with cognitively simple explanation rules? Table 3 shows the performance of explanation rules with FG features and metafeatures for the LR models. One of the first key questions related to the performance of the rules is "what is the fidelity", as we want our explanations to mimic the black-box as closely as possible. Overall, the results indicate that the fidelity is higher for DDMF than for the FG-based rules (on average, across all data sets, 6.05%). The rules with DDMF achieve a higher number of wins for both the fidelity and f-fidel (8 in contrast to 1). We use a one-tailed Wilcoxon signed-rank test [22] to make a statistical comparison between the fidelity of rules with DDMF vs FG features. The test is performed with a sample size of 9 data sets. We find a test statistic T =2 (which is smaller than the critical value T c =3), hence the difference in fidelity between DDMF and FG is statistically significant at a 1% significance level. The difference in f-fidel is statistically significant at a 5% level (test statistic of T =5 compared to a critical value of T c =8).
One notable exception is the 20news data: the FG-based rules outperform the DDMF-based rules, and the fidelity values are very high while the f-fidel results are comparably low. This is likely because of the severe (predicted) class imbalance (b=4.24% in Table 1) compared to the other data. For this reason, the fidelity criterion might be less suitable for this specific data set. Instead, we could have optimized the depth of the tree and the k of the DDMF on the f-fidel as measured on the validation set. 21 In order to better understand what drives some of the differences in the performance of explanations with FG features and DDMF, we conjecture this relates to the information held at and the coverage of the most predictive features 22 . We look at the Gini impurity reduction (used by the CART algorithm) for different features 23 , which we plot in Fig. 8 and 9 in Appendix 6. The results (visually) indicate that the ratio in Gini impurity reduction of the top-ranked metafeatures and the FG features relate to the difference in fidelity between rules using FG and DDMF features. For example, consider again the Flickr data, for which the explanation rules with metafeatures achieve a fidelity of 20.46% higher compared to the FG rules. From Fig. 8(g) we observe that the top-DDMF holds much more information (larger Gini impurity reduction) than the top-FG feature, which might explain the large fidelity difference between the explanations. Indeed, the correlation coefficient between this ratio (impurity reduction of top-ranked DDMF vs top-ranked FG) and the difference in fidelity between explanations with DDMF vs FG (from Table 3) is 0.81.
Secondly, moving to the stability of the explanations, we observe from Table 3 that the rules with DDMF are similar in stability compared to the FG features (5 wins vs 4). The difference in stability is not statistically significant. It is important to note that for the DDMF-based explanations, there are two sources of instability: computing metafeatures from different bootstrap samples, and extracting explanation rules for different bootstrap samples. When we would "fix" the data-driven metafeatures, and not compute them for different bootstrap samples, the stability of the DDMF-based explanations increases, and is comparable to the DomainMF-based explanations. Furthermore, the stability results can be closely tied to the parameter k. When the optimal dimensionality k of the DDMF is low (for example Movielens1m and Tafeng), the same DDMF are likely to appear in the global explanation, resulting in more stable explanations over the bootstrap samples. When the selected value of k is higher (for example 20news and Airline), the stability of the explanations with DDMF decreases.
Thirdly, when we compare the accuracy between the rules with DDMF and FG, we observe that the metafeatures-based explanations result in more accurate predictions in regard to the true labels y (7 wins versus 2). However, using a Wilcoxon signed-rank test, we find that the difference in accuracy is not significant at a 5% level. One data set that stands out is Libimseti. For this data set, the fidelity and accuracy for DDMF-based explanations as compared to FG-based explanations is respectively better and worse. Stronger even: the accuracies of the explanations with FG, DDMF and DDMF-SVD (94.38%, 87.94% and 99.63%) are better compared to the accuracy of the black-box model (82.71% in Table 2). Despite the sparsity of this data, there are features that have a large coverage and that are very predictive in regard to the (predicted) target values. For Libimseti, there exists a prediction model that has a small number of features (e.g., tree-based model with a depth of 5) that is more accurate compared to models on the full set of behavioral features. As a consequence, this seems not to be a problem instance that requires post-hoc explanations using rule-extraction. This example illustrates that one should always carefully verify first that there are black-box models on the full behavioral or textual data that, indeed, outperform intrinsically-interpretable models (e.g., small decision trees or linear models with a small number of features). If not, it may not help to use a black-box model and then compute post-hoc explanations from it [73]. Leaving out the Libimseti data and performing the Wilcoxon test on the eight remaining data sets, we find that the differences in fidelity and accuracy between DDMF and FG explanations are statistically significant at a 5% level.
Instead of generating metafeatures using a data-driven method, we can also rely on domain-based metafeatures, crafted by experts. The prominent advantage of this approach is that the resulting metafeatures are (by design) comprehensible. However, they may not always be available. For example, we have such metafeatures for only two of the nine data sets: Facebook and Movielens1m. When comparing DDMF with domain-based metafeatures for these two data sets, we see again that the fidelity is higher for the DDMF compared to the DomainMF ( Table 3 shows that the rules with domain-based metafeatures achieve, at best, test fidelities of 77.66% for Facebook and 71.47% for Movielens1m), providing further support for using DDMF when developing global explanations for black-boxes. However, when a straightforward semantic meaning of the metafeatures is key, one might still prefer to use DomainMF if they can also increase the fidelity relative to the explanation with FG features (for example for the Facebook data). 21 However, for simplicity, we only used fidelity for all data sets. 22 The features are either the fine-grained behavioral or textual features, or the metafeatures. With "predictive" we mean predictive in regard to the predicted labels of the black-box model. 23 We compute the average Gini impurity of the top-FG features and top-MF over five folds.
6.2 How does the fidelity of explanation rules vary over different complexity settings? Figure 2: Difference in average test fidelity of rules with DDMF and FG features in percentage points for varying complexity settings (tree depths from 1 to 5). Fig. 2 plots the difference in average test fidelity between rules with DDMF and rules with FG features against the maximum allowed explanation complexity (equivalent to the tree depth). Points above the horizontal line are data sets for which the rules with DDMF perform better. The graph clearly shows that for the majority of data, and over varying complexity settings, the DDMF representation performs better than the FG (differences larger than 0). Only for the Tafeng, Airline and 20news data, the differences are sometimes not positive, indicating that for these complexity settings, the average test fidelity for the rules with FG features is best. In general, from this plot, we can conclude that the findings of Table 3 hold for varying complexity settings, and that the fidelity is generally higher for explanations with the DDMF representation compared to the FG representation. Fig. 3 plots the average test fidelity against the maximal allowed explanation complexity for FG (10a) and DDMF (10b) explanation rules. We observe that, as one would expect, for all data sets, there is generally an increasing fidelity when we increase the depth of the decision tree, or equivalently, the complexity of the explanation rules. Interestingly, for some data sets, this fidelity-complexity trade-off is less severe. For example, for the 20news and Movielens100 data, the slopes of the curves are relatively flat. These results also indicate that in some cases, there may not be much to gain by using a relatively "more complex" explanation. Therefore, once one is willing to trade-off fidelity for complexity, in some cases, one might as well choose an "extremely" simple explanation. For the 20news data, we already pointed at the f-fidel being a more suitable measure than fidelity because of the class imbalance, which might explain the marginal increase in fidelity when increasing complexity.
6.3 How does the number of generated data-driven metafeatures (dimensionality reduction parameter k) impact fidelity and stability?
A key parameter in our metafeatures-based rule-extraction methodology is the dimensionality reduction parameter k. For DomainMF, this k is fixed. For DDMF, we have been selecting the value of k that maximizes the fidelity of the explanation rules on the validation data. As this k may be an important parameter that defines the dimensionality of the space where rule-extraction methods operate (and their performance), we also investigate to what extent the quality -both fidelity and stability -of rules extracted using DDMF depends on this parameter. 24 Although fidelity can be considered the most important evaluation metric, in practice, one may wish to tune parameters such as k on a desired combination of fidelity, stability and accuracy 25 depending on the context. Firstly, looking at the figures depicted on the left, we observe that, for most data, the fidelity increases with a higher 24 One can do such an analysis for other parameters, too, in general. 25 As mentioned earlier, we focus on fidelity -namely how well we can mimic the black-box -instead of accuracy. All analyses can be done for either of the two -or for both -although trade-off decisions become more complex when one uses many criteria. number of metafeatures up until a certain point, after which fidelity decreases again. This turnover point varies per data set, and also depends on the complexity of the explanation (the tree depth). Therefore, an important implication is to select the optimal number of metafeatures on a separate validation set, as we also do. Interestingly, fidelity behaves similarly to how (out-of-sample) accuracy typically does as complexity increases: for both measures there is some sort of "overfitting" to the black-box training data in case of including too many metafeatures.
On the other hand, for stability (all figures shown on the right), we observe that, overall, the stability of the extracted rules decreases with a higher number of k, especially when allowing for a larger explanation complexity. For example, the stability of rules with DDMF with k=10 is generally larger than the stability with DDMF with k=700. For a lower value of k, the dimensionality and sparsity of the metafeatures are lower, making the same metafeature more likely to appear in explanations from different bootstrap samples (as also explained in Section 6.1). Interestingly, these figures also show that there is a fidelity-stability trade-off. While fidelity generally increases (at first) with the number of metafeatures and the explanation complexity, stability does not. This may also impact the "optimal" number of generated metafeatures k, or any parameter selection for any explanation methodology.

Conclusion
The fine-grained level of the features that are typically observed in behavioral and textual data sets are of great value for predictive modeling. Feature selection methods or dimensionality reduction techniques to come to a reduced set of "metafeatures" have been shown in the literature to lead to lower accuracies [41,15,21] for models mining these types of data. On the other hand, we have shown empirically using a number of data sets, and for Logistic Regression as black-box model, that these metafeatures are of great value to explain the complex prediction models built on the fine-grained features. The results indicate that explanation rules extracted with data-driven metafeatures are better able to mimic the black-box models than those extracted using the fine-grained features. As such, metafeatures help to improve the fidelity: concise rule sets that explain a large(r) percentage of the black-box's predictions (higher fidelity) can be obtained.
Our empirical results also show important trade-offs between the quality measures of the explanation rules that we considered. For example, more complex explanations (larger rule sets) tend to lead to higher fidelity but lower stability. An interesting implication of our empirical findings is that one should carefully fine tune any parameter of their explainability method, such as the number of generated metafeatures in our methodology, in order to obtain the desired trade-offs. In our case, increasing the number of generated metafeatures has shown to result in lower stability of the extracted rules, whereas the impact on fidelity is not straightforward and depends on the data set and the complexity setting.
In this article, we mainly focused on the fidelity of explanation rules in regard to the black-box model. For future research, there are some other important directions to explore for evaluating post-hoc explanations of prediction models: the computational cost to achieve the explanations, the cost of having an explanation rule set with an accuracy that is lower than the black-box model, or the issue of presenting only one rule set as explanation, while other rule sets with similar fidelity and accuracy might exist. Although these aspects are implicitly addressed in our article, a more qualitative study on how these "costs" are perceived by users can be another interesting issue for future research. On a methodological level, this study could spur future research on the use of other feature engineering techniques such as embeddings to be used in metafeatures-based explanation rules. One interesting approach is to include the fidelity, stability, accuracy, and complexity measures explicitly when constructing the metafeatures.
Finally, our metafeatures-based explanation approach for high-dimensional, sparse behavioral and textual data has important practical implications for any setting where such data is available and explainability is an important requirement, be it for model acceptance, validation, insight, or improvement. This article could therefore potentially lead to a wider use of valuable behavioral and textual data in domains like marketing and fraud detection among others.
8 Appendix 8.1 Appendix 1: Procedure for generating data-driven metafeatures. Figure 6: Procedure for generating DDMF using dimensionality reduction with matrix factorization. Note that XF G can also contain numerical features.