{'fraction_non_alphanumeric': 0.03383241716524433, 'fraction_numerical': 0.010568834087526954, 'mean_word_length': 4.839333215109062, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Neural Class Expression Synthesis

Most existing approaches for class expression learning in description logics are search algorithms. As the search space of these approaches is infinite, they often fail to scale to large learning problems. Our main intuition is that class expression learning can be regarded as a translation problem. Based thereupon, we propose a new family of class expression learning approaches which we dub neural class expression synthesis. Instances of this new family circumvent the high search costs entailed by current algorithms by translating training examples into class expressions in a fashion akin to machine translation solutions. Consequently, they are not subject to the runtime limitations of search-based approaches post training. We study three instances of this novel family of approaches to synthesize class expressions from sets of positive and negative examples. An evaluation of our approach on four benchmark datasets suggests that it can effectively synthesize high-quality class expressions with respect to the input examples in approximately one second on average. Moreover, a comparison to other state-of-the-art approaches suggests that we achieve better F-measures on large datasets. For reproducibility purposes, we provide our implementation as well as pretrained models in our public GitHub repository at https://github.com/fosterreproducibleresearch/NCES


INTRODUCTION
One of the most popular families of web-scale knowledge bases [16] is that of RDF knowledge bases equipped with an ontology in W3C's web ontology language OWL [28]. Examples include DBpedia [3], Wikidata [40], and CaliGraph [17]. One means to implement antehoc explainable machine learning on these knowledge bases is class expression learning (also called concept learning) [13,15,24,26,34,39]. Informally, class expression learning approaches learn a class expression which describes individuals provided as positive examples. Class expression learning has applications in several domains, including ontology engineering [25], bio-medicine [27] and Industry 4.0 [2]. There exist three main learning settings in class expression learning: (1) positive and negative learning, (2) positiveonly learning, and (3) class-inclusion learning [23]. This paper tackles setting (1).
Several methods have been proposed to solve class expression learning problems, the best are based on refinement operators [13,20,23,24,26,34] and evolutionary algorithms [15]. A common drawback of these approaches is their lack of scalability. While the reasoning complexity of all learning approaches grows with the expressivity of the underlying description logic (DL) [19,32], those based on refinement operators and evolutionary algorithms further suffer from the exploration of an infinite conceptual space for each learning problem [34]. Another inherent limitation of existing methods for class expression learning from examples is their inability to leverage previously solved problems-their algorithm always starts from scratch for each new learning problem.
In view of the large sizes of modern knowledge bases, e.g., DBpedia [3] and Wikidata [40], we propose a new family of approaches, dubbed neural class expression synthesizers (NCES), for webscale applications of class expression learning. The fundamental hypothesis behind this family of algorithms is that one should be able to capture enough semantics from latent representations (e.g., embeddings) of examples to directly synthesize class expressions in a fashion akin to machine translation, i.e., without the need for costly exploration. This hypothesis is supported by the significant improvement in the performance of machine translation approaches arXiv:2111.08486v3 [cs.AI] 19 Dec 2022 brought about by neural machine translation (NMT) [7,44]. NMT approaches translate from a source language to a target language by exploiting an intermediary representation of a text's semantics. NCES behave similarly but translate from the "language" of sets of positive/negative examples to the "language" of class expressions. We instantiate this new paradigm by implementing three NCES instances that target the description logic ALC. We show that our NCES instances generate high-quality class expressions with respect to the given sets of examples while remaining scalable. Note that NCES instances synthesize solutions for multiple learning problems at the same time as they accept batches of inputs. This makes NCES particularly fit for deployment in large-scale applications of class expression learning, e.g., on the web.
The rest of the paper is organized as follows: First, we present existing approaches for class expression learning and introduce the notations and prerequisites needed throughout the paper. Next, we describe the intuition behind NCES in detail and introduce three instantiations of this new family of algorithms. We then compare these instantiations with state-of-the-art approaches on four benchmark datasets. Finally, we discuss our results and draw conclusions from our experiments.

RELATED WORK
Class expression learning has been of interest to many researchers in recent years. Of the proposed approaches, the most prominent include those based on evolutionary algorithms [15] and refinement operators [6,13,20,24,25,36]. The state-of-the-art EvoLearner [15] initializes its population by random walks on the knowledge graph which are subsequently converted to description logic concepts, represented as abstract syntax trees. These concepts are further refined by means of mutation and crossover operations. EvoLearner outperformed approaches based on refinement operators such as CELOE and OCEL from the DL-Learner framework [15]. Previously, Lehmann and Hitzler [26] studied different properties that a refinement operator can have, then designed a refinement operator to learn class expressions in description logics. Their learning algorithm, CELOE [25], is implemented in DL-Learner [23] alongside OCEL and ELTL [6]. CELOE extends upon OCEL by using a different heuristic function, and it is currently regarded as the best class expression learning algorithm in DL-Learner. Although ELTL was designed for the lightweight description logic EL, we include it in this study to check whether our generated learning problems can be solved in a simpler description logic. ECII [36] is a recent approach for class expression learning that does not use a refinement operator and only invokes a reasoner once for each run. This approach was designed to overcome the runtime limitations of refinement operator-based approaches. Other attempts to prune the search space of refinement operator-based approaches include DL-Focl [34]. It is a modified version of DL-Foil [13] that is quintessentially based on omission rates. DL-Focl uses techniques such as lookahead strategy and local memory to avoid reconsidering sub-optimal choices.
Even though existing approaches for class expression learning have shown promising results, most of them are search-based. As a result, these approaches often use entailment checks-which are hard to compute, see Ozaki [32]-or compute classification accuracies at each step of the search process. In contrast, NCES instances synthesize class expressions without a search process. As the training process of our synthesizers only involves instance data embeddings and a vocabulary of atoms, they can be extended to more expressive description logics such as ALCQ (D).

BACKGROUND 3.1 Notation
DL is short for description logic, and DNN stands for deep neural network. Unless otherwise specified, K = (TBox, ABox) is a knowledge base in ALC, and N is the set of all individuals in K. The ABox consists of statements of the form ( ) and ( , ), whereas the TBox contains statements of the form ⊑ , where , are concepts, is a role, and , are individuals in K. We use the representation of OWL knowledge bases as sets of triples to compute embeddings of individuals, classes and roles. The conversion into triples is carried out using standard libraries such as RDFLib [21]. We then use the knowledge graph representation of the knowledge bases to compute embeddings, which are essential to our proposed approach (see Figure 1). The function |.| returns the cardinality of a set. 1 denotes the indicator function, i.e., a function that takes two inputs and returns 1 if they are equal, and 0 otherwise. Let a matrix and integers , be given. :, , ,: , and represent the -th column, the -th row, and the entry at the -th row and -th column, respectively. Similar notations are used for higher-dimensional tensors.
Vocab[ ] is the atom at position in Vocab. These constructs are used by NCES to synthesize class expressions in ALC (see Section 4 for details). Let be a class expression, thenˆand¯are the list (in the order they appear in ) and set of atoms in , respectively.

Description Logics
Description logics [30] are a family of knowledge representation paradigms based on first-order logics. They have applications in several domains, including artificial intelligence, the semantic web, and biomedical informatics. In fact, the web ontology language, OWL, uses description logics to represent the terminological box of RDF ontologies. In this work, we focus on the description logic ALC (Attributive Language with Complement) [37] because of its simplicity and expressiveness. The syntax and semantics of ALC are presented in Table 1.
A refinement operator can be finite, proper, redundant, minimal, complete, weakly complete or ideal. Note that some of these properties can be combined whilst others cannot [26]. For class expression
Most existing approaches use hard-coded heuristics or refinement operators to search for the solution . When an exact solution does not exist, an approximate solution in terms of accuracy or F-measure is to be returned by the approaches. In this work, we exploit the semantics embedded in latent representations of individuals to directly synthesize .

Knowledge Graph Embedding
A knowledge graph can be regarded as a collection of assertions in the form of subject-predicate-object triples ( , , ). Embedding functions project knowledge graphs onto continuous vector spaces to facilitate downstream tasks such as link prediction [5], recommender systems [47], and structured machine learning [20]. Many embedding approaches for knowledge graphs exist [9,41]. Some of them use only facts observed in the knowledge graph [4,31]. Others leverage additional available information about entities and relations, such as textual descriptions [42,45]. Most embedding approaches initialize each entity and relation with a random vector, matrix or tensor and learn the embeddings as an optimization problem. For example, TransE [5] represents entities and relations as vectors in the same space and aims to minimize the Euclidean distance between + and for each triple ( , , ). In this work, we use two embedding approaches to evaluate our approach and conduct additional experiments (see Section 4.2 and Section A.2).

Permutation-Invariant Neural Network Architectures for Set Inputs
We deal with set-structured input data as in 3D shape recognition [33], multiple instance learning [11], and few-shot learning [14,38]. These tasks benefit from machine learning models that produce the same results for any arbitrary reordering of the elements in the input set. Another desirable property of these models is the ability to handle sets of arbitrary size. In recent years, several approaches have been developed to meet the aforementioned requirements. The most prominent of these approaches include Deep Set [46] and Set Transformer [22]. The Deep Set architecture encodes each element in the input set independently and uses a pooling layer, e.g., averaging, to produce the final representation of the set. In contrast, the Set Transformer architecture uses a self-attention mechanism to represent the set, which allows pair-wise and even higher-order interactions between the elements of the input set. As a result, the Set Transformer architecture shows superior performance on most tasks compared to Deep Set with a comparable model size [22]. In this work, we hence choose the Set Transformer architecture (more details in Section 4) whose building blocks are elucidated in the following paragraphs.
Attention. Given query, key, value vectors ∈ R × , ∈ R × , ∈ R × , and an activation function , an attention function computes the output as a weighted sum of : Here, , , , are the numbers of queries, values, dimensions of each query vector, and dimensions of each value vector, respectively. The activation function is chosen in a way that higher weights correspond to higher dot product values between queries and keys. We use softmax as activation function, as in [22].
Multi-head Attention. A multi-head attention function with ℎ heads first projects , , onto ℎ different projections , , for = 1, . . . , ℎ. Each projection is performed using a trainable projection map, usually a linear layer. Next, the attention function Att(.; ) is applied to each projection, and the outputs are concatenated. Finally, the result of the concatenation is fed to a linear layer for the final output: For computational efficiency, a natural choice for the projection dimensions is Multi-head Attention Block (MAB). Lee et al. [22] introduced a modified version of multi-head attention blocks tailored toward set-structured inputs. We employ the same attention blocks but without layer normalisation, as the latter did not improve the results in our learning tasks. For any , ∈ R × , we define MAB as: where rFF is a feedforward neural network layer and is the number of elements in the input sets of embeddings.
Set Attention Block (SAB). This module operates on a single set of embedding vectors. Following [22], we define SAB as: Induced Set Attention Block (ISAB). To reduce the computational complexity in attention mechanisms for especially large sets, [22] proposed using so-called inducing points. The main idea is to introduce a (small) fixed-size trainable matrix ∈ R × as a bridge in SAB. More specifically, ISAB is defined for any ∈ R × as: Note that the computation cost in ISAB is 4 × ( × × ). This is less than that of SAB 2 × ( × × ) for >> . In this paper, we learn 32 inducing points ( = 32).
Pooling by Multi-head Attention (PMA). As opposed to the pooling mechanism in [46], where the learned features from the encoder are summed (decoding), the decoder in Set Transformer applies a multi-head attention function between a trainable feature matrix and the features from the encoder. Formally, a Pooling by Multihead Attention layer (PMA) with seed vectors ∈ R × is defined for any ∈ R × by: In this work, we learn a single seed vector ( = 1).

NEURAL CLASS EXPRESSION SYNTHESIS
In this section, we present our proposed family of approaches for class expression learning from examples. We start with a formal definition of the learning problem that we aim to solve, then present our proposed approach in detail. The above definition implies the need for a system that can map instance data to class expressions in the description logic ALC. We devise such a system by building and training well-selected neural network architectures. The system and its components are referred to as neural class expression synthesizers (NCES).

Learning Approach
We propose the following simple recipe to implement the idea behind NCES. First, given a knowledge base over ALC, convert it into a knowledge graph (see Subsection 3.1). Then, embed said knowledge graph into a continuous vector space using any stateof-the-art embedding model in the literature. We used ConEx [10] for this purpose because it outperforms many other embedding models while remaining scalable. Note that we also used TransE in additional experiments to further validate the effectiveness of our proposed approach (see Appendix A.2). The computed embeddings are then used as features for a model able to take a set of embeddings as input and encode a sequence of atoms as output (see Figure 1).

Neural Network Architectures.
We conduct our experiments using the following network architectures: the Long Short-Term Memory (LSTM) [18], the Gated Recurrent Unit (GRU) [8], and the Set Transformer [22]. The latter is known to be permutation equivariant while the two others are not (see more in Section 5). Nonetheless, LSTM and GRU can handle set inputs as long as an ordering is defined since they deal well with sequential data [49,50]. In this work, we use the default ordering (the order in which we received the data) of the elements in each set during the data generation process (see Section 5.1.2).
Recurrent Networks (LSTM and GRU). We use two recurrent layers followed by three linear layers with the relu activation function and a batch normalization layer. A recurrent neural network produces a sequence of hidden states ℎ ( =1, . . . , ) for each input sequence of length . In this work, we are concerned with a sequence of 1 Here, is the relu activation function, bn is a batch normalization layer, and 1 , 1 , 2 , 2 , 3 , 3 are trainable weights.
Set Transformer. This architecture comprises an encoder Enc and a decoder Dec. The encoder is a stack of two ISAB layers, and the decoder is composed of a single PMA layer with one seed vector ( = 1), and a linear layer. As in the previous paragraph, the sets of positive and negative examples for a given class expression are first encoded separately using the encoder. The outputs are then concatenated row-wise and fed to the decoder: = Dec(Concat( , )).
Although features through self-attention. This demonstrates the representational power of the Set Transformer model for our set-structured inputs for class expression synthesis.
The output from 11 and 13 is reshaped into a (1 + |Vocab|) × matrix, where is the length of the longest class expression that NCES instances can generate. These scores allow us to compute the loss (see Equation 15) and update model weights through gradient descent during training.

Loss.
We train our NCES instances using the the loss function L defined by: where is the size of the minibatch, is the number of classes, ∈ R × × is the minibatch of predicted class scores for each position in the target sequence of atoms, and ∈ N × is the minibatch of actual class indices. Minimizing L constrains the model to assign a high score to the entry corresponding to the correct token (exp( , , ) ≈ 1) while keeping the remaining scores relatively low ( =1, ≠ exp( , , ) ≈ 0). In this work, = 1 + |Vocab|, where the additional +1 accounts for the special token "PAD" that we used to pad all class expressions to the same length. Contrarily to some works that omit this special token when computing the loss, we use it as an ordinary token during training. This way, we can generate class expressions more efficiently at test time with a single forward pass in the model, then strip off the generated tokens after the special token. To avoid exploding gradients and accelerate convergence during training, we adopt the gradient clipping technique [48].

Learning Metrics.
Apart from the loss function, we introduce two accuracy measures to quantify how well neural networks learn during training: soft accuracy and hard accuracy. The former only accounts for the correct selection of the atoms in the target expression, while the latter additionally measures the correct ordering of the selected atoms. Formally, let and be the target and predicted class expressions, respectively. Recall the notation¯andˆi ntroduced in Section 3.1 for any class expression . The soft ( ) accuracy and hard accuracy ( ℎ ) are defined as follows: where 1 and 2 are the lengths ofˆandˆ, respectively.

Model Ensembling.
Ensemble learning has proven to be one of the most robust approaches for tasks involving complex noisy data [12,35]. In this work, we combine class expression synthesizers' predictions post training by averaging the predicted scores. Specifically, given the output scores ∈ R × ( = 1, 2, 3) as defined in 11 and 13 for the three models LSTM, GRU, and Set Transformer, we consider four different ensemble models: three pairwise ensemble models, and one global ensemble model (LSTM, GRU, and Set Transformer are combined). Formally, the ensemble scores are computed as: with I ⊆ {1, 2, 3} and |I| ≥ 2.
Then, the synthesized expression is constructed as explained in Section 4.2.4 using the average scores .  Table 2.

Training and Test Data Construction.
We generated class expressions of different forms from the input knowledge base using the recent refinement operator by [20] that was developed to efficiently generate numerous class expressions to serve as training data for concept length prediction in ALC. The data that we generate is passed to the filtering process, which discards any class expression such that an equivalent but shorter class expression was not discarded. Note that each class expression comes with its set of instances, which are computed using the fast closed-world reasoner based on set operations described in [15]. These instances are considered positive examples for the corresponding class expression; negative examples are the rest of the individuals in the knowledge base. Next, the resulting data is randomly split into training and test sets. To ensure that our approach is scalable to large knowledge bases, we introduce a hyper-parameter = 1 + 2 that represents the total number of positive and negative examples we sample for each class expression to be learned by NCES. Note that is fixed for each knowledge base, and it depends on the total number of individuals.

Evaluation Metrics.
We measure the quality of a predicted class expression in terms of accuracy and F-measure with respect to the positive/negative examples. Note that we cannot expect to exactly predict the target class expression in the test data since there can be multiple equivalent class expressions.

Hyper-parameter Optimization for NCES.
We employed random search on the hyper-parameter space since it often yields good results while being computationally more efficient than grid search [1]; the selected values-those with the best results-are reported in Table 3. In the table, it can be seen that most knowledge bases share the same optimal values of hyper-parameters: the minibatch size , the number of training epochs ℎ , the optimizer ., the learning rate , the maximum output sequence length , the number of embedding dimensions , the number of inducing points , and the gradient clipping value . Although we may increase for very large knowledge bases, = min( | N | 2 , 1000) appears to work well with our evaluation datasets. This suggests that one can effortlessly find fitting hyper-parameters for new datasets.

Hardware and Training Time.
We trained our chosen NCES instances on a server with 1TB of RAM and an NVIDIA RTX A5000 GPU with 24 GB of RAM. Note that during training, approximately 8GB of the 1TB RAM is currently used by NCES. As search-based approaches do not require a GPU for class expression learning, we used a 16-core Intel Xeon E5-2695 with 2.30GHz and 16GB RAM to run all approaches (including NCES post training) for class expression learning on the test set. The number of parameters and training time of each NCES instance are reported in Table 4. From the table, we can observe that NCES instances are lightweight and can be trained within an hour on medium size knowledge bases. Note that training is only required once per knowledge base.

Syntactic Accuracy.
Our neural class expression synthesizers were trained for 500 epochs on each knowledge base. In Figure  2, we only show the hard accuracy curves during training due to space constraints. The rest of the training curves can be found on our Github repository. The curves in Figure 2 suggest that NCES instances train fast with exponential growth in accuracy for the first 10 epochs. All models reach a syntactic accuracy of 100% between 400 and 500 training epochs on large knowledge bases (Carcinogenesis and Vicodi) while achieving a lower performance on smaller ones (Mutagenesis and Semantic Bible). This suggests that NCES instances are well suited for large datasets. We validate this hypothesis through the quality of the synthesized solutions on the test set (see Table 5).

Comparison to
State-of-the-art Approaches. We compare our approach against EvoLearner, CELOE, ECII, ELTL, and DL-Foil. Note that DL-Foil is not included in Table 5 because it could not terminate on most learning problems that we considered. The algorithm actually gets stuck on the refinement of a partial description during the search process on some learning problems. On the other side, we could not run DL-Focl with the documentation provided. The maximum execution time for CELOE and EvoLearner was set to 300 seconds per learning problem while ECII and ELTL were executed with their default settings, as they do not have the maximum execution time parameter in their original implementation. From Table 5, we can observe that our approach (with ensemble prediction) significantly outperforms all other approaches in runtime on all datasets, and in accuracy and F-measure on the Carcinogenesis, Mutagenesis, and Vicodi knowledge bases. We used the Wilcoxon Rank Sum test with a significance level of 5% and the null hypothesis that the compared quantities per dataset are from the same distribution. The best search-based approach only outperforms NCES instances (including ensemble models) on the smallest dataset, the Semantic Bible knowledge base. The reason Table 2: Detailed information about the datasets used for evaluation. |Vocab| is computed as |Classes| + |Prop.| + 11, where +11 accounts for the special characters or atoms (see caption of Figure 1). |LPs| is the number of learning problems in the test set.   for this is that deep learning models are data-hungry and often fail to generalize well on small datasets. Our approach is hence well suited for large knowledge bases where search-based approaches are prohibitively slow.

Predictive Performance Analysis of NCES Instances.
To better understand the main differences between our selected NCES instances, we compared their synthesized solutions on the most challenging learning problems. These learning problems are those for which at least an NCES instance synthesized a solution with an F-measure less than 10%. The results are reported in Figure 3.
The size of each point in the figure is proportional to the number of times the corresponding NCES instance was ranked first in terms of F-measure. From the figure, we can observe that the Set Transformer model outperforms recurrent neural networks on three datasets: Mutagenesis, Semantic Bible, and Vicodi (see the size of the blue points), while ranking second on the Carcinogenesis knowledge base behind the GRU model. It can also be observed that for each of these learning problems, there always exist at least one NCES instance that synthesizes a solution with quality of at least 50%. This motivated us to ensemble different NCES instances, which indeed led to increased predictive performance (see Table 5).

Discussion
The hypothesis behind this work was that high-quality class expressions can be synthesized directly out of training data, i.e., without the need for an extensive search. Our results clearly undergird our hypothesis. While our approaches are outperformed by CELOE on the Semantic Bible knowledge base, they achieve the best performance on the three other datasets with over 11.92% absolute improvement on the Carcinogenesis dataset. This large difference is due to the fact that most search-based approaches fail to find any suitable solution for some learning problems. The scalability of the synthesis step of our approaches makes them particularly suitable for situations where many class expressions are to be computed for the same knowledge base. For example, using the Set Transformer architecture on the Carcinogenesis knowledge base, one can conjecture that the minimum number of learning problems from which the cost of deep learning becomes worthwhile is: 13 for NCES vs. CELOE, 69 for NCES vs. EvoLearner, 114 for NCES vs. ELTL, and 1 for NCES vs. ECII. These values are calculated by solving for in × algo_learn > train + inference , where algo_learn , train , and inference are the average learning time of a search-based approach, the training time, and the inference time of NCES, respectively. Although ELTL appears to be one of the fastest search-based approaches, it computes lower quality solutions compared to the other approaches as it runs in the lightweight description logic EL.

CONCLUSION AND FUTURE WORK
We proposed a novel family of approaches for class expression learning, which we dub neural class expression synthesizers (NCES).  Table 5: Evaluation results per approach and dataset. The star (*) indicates statistically significant differences between the best search-based and the best synthesis-based approaches. We report the mean and standard deviation. ↑ indicates that the higher is better (F-measure and Accuracy), and ↓ indicates that the lower is better (Runtime). Underlined values are the second best. where many concepts are to be learned for the same knowledge base. In future work, we will investigate means to transfer the knowledge acquired on one knowledge base to other knowledge bases. Furthermore, we plan to extend our approach to more expressive description logics such as ALCQ (D).

A APPENDIX
Additional experiments were conducted to further demonstrate the effectiveness of neural class expression synthesizers. These include performance measures with respect to random permutations of the elements in the input sets, and class expression synthesis using the simple embedding model TransE.

A.1 Stability with Respect to Shuffled Inputs
We compared all three neural network architectures with respect to random permutations applied to the sets of positive and negative examples. First, all models are trained with the default ordering  in the input sets. Then, they are evaluated on the unseen learning problems where random permutations are applied to the elements in the input sets. The results are reported in Table 6. From the table, we can observe that shuffling the input sets of examples does not have any impact on the output of the Set Transformer model. Oppositely, we can observe significant changes in the quality of the synthesized expressions for recurrent neural network models (LSTM and GRU). The GRU model showed improvements in Fmeasure on three datasets (with up to 19.06% on Mutagenesis) and dropped by 9.78% F-measure on the Semantic Bible knowledge base. The LSTM model synthesized poorer solutions on three out of four datasets with a narrow improvement by 1.96% F-measure on the Mutagenesis knowledge base. The differences in accuracy are smaller because this metric does not always properly reflect the quality of the synthesized solutions. From these results, it is clear that the Set Transformer architecture is best suited for class expression learning since it yields the same predictions regardless of the order of the elements in the input sets. Nonetheless, recurrent neural network architectures can be useful in application scenarios where one needs multiple, closely related predictions for the same input sets.

A.2 Class Expression Synthesis Using TransE Embeddings
To check whether our approach works only with specific types of embedding models, we ran additional experiments using the TransE [5] embedding model. TransE is a translational embedding model that represents entities and relations as vectors in the same space and aims to minimize the Euclidean distance between + and for each triple ( , , ) in the input knowledge graph. Although it is a simple embedding model, TransE remains one of the most popular embedding models to achieve state-of-the-art results in many downstream tasks while remaining scalable to large datasets. Figure 4 shows the training curves of NCES instances using TransE embeddings. As in the case of ConEx embeddings, the training accuracy curves grow exponentially within the first 10 epochs and reach their culminant point (100%) during training on the Carcinogenesis and Vicodi knowledge bases. Similarly, the loss decreases at the same rate and approaches 0 in the last training epochs. Additionally, Table 7 shows that high-quality solutions are synthesized on the test set. This suggests that our NCES instances are able to learn how to map instance data to class expressions in ALC with translational embeddings provided by TransE. Consequently, one can expect NCES instances to work well with other embedding models provided that they compute embeddings that are to some extent faithful to the input knowledge graph.