{'fraction_non_alphanumeric': 0.044434425408275725, 'fraction_numerical': 0.011722272317403066, 'mean_word_length': 3.8289791969037252, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Operational State Complexity of Deterministic Unranked Tree Automata

We consider the state complexity of basic operations on tree languages recognized by deterministic unranked tree automata. For the operations of union and intersection the upper and lower bounds of both weakly and strongly deterministic tree automata are obtained. For tree concatenation we establish a tight upper bound that is of a different order than the known state complexity of concatenation of regular string languages. We show that (n+1) ( (m+1)2^n-2^(n-1) )-1 vertical states are sufficient, and necessary in the worst case, to recognize the concatenation of tree languages recognized by (strongly or weakly) deterministic automata with, respectively, m and n vertical states.


Introduction
As XML [1] has played increasingly important roles in data representation and exchange through the web, tree automata have gained renewed interest, particularly tree automata operating on unranked trees. XML documents can be abstracted as unranked trees, which makes unranked tree automata a natural and fundamental model for various XML processing tasks [2,9,13]. Both deterministic and nondeterministic unranked tree automata have been studied.
One method to handle unranked trees is to encode them as ranked trees and then use the classical theory of ranked tree automata. However, the encoding may result in trees of unbounded height since there is no a priori restriction on the number of the children of a node in unranked trees. Also depending on various applications, it may be difficult to come up with a proper choice of the encoding method.
Descriptional complexity of finite automata and related structures has been extensively studied in recent years [5,6,7,14,15]. Here we consider operational state complexity of deterministic unranked tree automata. Operational state complexity describes how the size of an automaton varies under regularity preserving operations. The corresponding results for string languages are well known [8,14,16], however, very few results have been obtained for tree automata. While state complexity results for tree automata operating on ranked trees are often similar to corresponding results on regular string automata [14], the situation becomes essentially different for automata operating on unranked trees. An unranked tree automaton has two different types of states, called horizontal and vertical states, respectively. There are also other automaton models that can be used to process unranked trees, such as nested word automata and stepwise tree automata. The state complexity of these models has been studied in [4,10,11].
We study two different models of determinism for unranked tree automata. We call the usual deterministic unranked tree automaton [2] model where the horizontal languages defining the transitions are specified by DFAs (deterministic finite automata), a weakly deterministic tree automaton (or WDTA). For the other variant of determinism for unranked tree automata, we refer to the corresponding automaton model as a strongly deterministic unranked tree automaton (or SDTA). This model was introduced by Cristau, Löding and Thomas [3], see also Raeymaekers and Bruynooghe [12]. SDTAs can be minimized efficiently and the minimal automaton is unique [3]. On the other hand, the minimization problem for WDTAs is NP-complete and the minimal automaton need not be unique [10].
We give upper and lower bounds for the numbers of both vertical and horizontal states for the operations of union and intersection. The upper bounds for vertical states are tight for both SDTAs and WDTAs. We also get upper bounds which are almost tight for the number of the horizontal states of SDTAs. Obtaining a matching lower bound for the horizontal states of WDTAs turns out to be very problematic. This is mainly because the minimal WDTA may not be unique and the minimization of WDTAs is intractable [10]. Also, the number of horizontal states of WDTAs can be reduced by adding vertical states, i.e., there can be trade-offs between the numbers of horizontal and vertical states, respectively.
The upper bounds for the number of vertical states for union and intersection of WDTAs and SDTAs are, as expected, similar to the upper bound for the corresponding operation on ordinary string automata. Already in the case of union and intersection, the upper bounds for the numbers horizontal states are dramatically different for WDTAs and SDTAs, respectively. In an SDTA, the horizontal language associated with label σ is represented with a single DFA H σ augmented with an output function λ . The state assigned to a node labeled with σ is determined by the final state reached in H σ and λ . On the other hand, in a WDTA, the horizontal languages associated with a given label σ and different states are represented by distinct DFAs. The state assigned to a node labeled with σ depends on the choice of the DFA.
We consider also the state complexity of (tree) concatenation of SDTAs. It is well known that m2 n − 2 n−1 states are sufficient to accept the concatenation of an m state DFA and an n state DFA [16]. However, the tight upper bound to accept the concatenation of unranked tree automata, with m and n vertical states respectively, turns out to be (n + 1)((m + 1)2 n − 2 n−1 ) − 1. The factor (n + 1) is necessary here because the automaton accepting the concatenation of two tree languages must keep track of the computations where no concatenation has been done. For string concatenation, there is only one path and the concatenation always takes place somewhere on that path. For non-unary trees, there is no way that the automaton can foretell on which branch the concatenation is done and, consequently, the automaton for concatenation needs considerably more states. It should be emphasized that this phenomenon is not caused by any particular construction used for the automaton to accept the concatenation of given tree languages, and we have a matching lower bound result.
Since complementation is an "easy" operation for both strongly and weakly deterministic tree automata, we do not investigate its state complexity in this paper. Note that we do not require the automaton models to be complete (i.e., some transitions may be undefined). A (strongly or weakly) deterministic automaton accepting the complement of a tree language recognized by the same type of automaton would need at most one additional vertical state and it is easy to see that this bound can be reached in the worst case.
The paper is organized as follows. Definitions of unranked tree automata and other notations are given in section 2. The upper bounds and corresponding lower bounds for union and intersection of SDTAs are presented in section 3.1. In section 3.2, the state complexity of union and intersection of WDTAs is discussed. The tight bound for the number of vertical states for tree concatenation of SDTAs is given in section 4. The same construction works for WDTAs.

Preliminaries
Here we briefly recall some notations and definitions concerning trees and tree automata. A general reference on tree automata is [2].
Let IN be the set of non-negative integers. A tree domain D is a finite set of elements in IN * with the following two properties: (i) If w ∈ D and u is a prefix of w then u ∈ D. (ii) If ui ∈ D, i ∈ IN and j < i then u j ∈ D. The nodes in an unranked tree t can be denoted by a tree domain dom(t), and t is a mapping from dom(t) to the set of labels Σ. The set of Σ-labeled trees is T Σ .
For t,t ′ ∈ T Σ and u ∈ dom(t ′ ), t ′ (u ← t) denotes the tree obtained from t ′ by replacing the subtree at node u by t. The concatenation of trees t and t ′ is defined as The concatenation operation is extended in the natural way to sets of trees L 1 , L 2 : We denote a tree t = b(a 1 , . . . , a n ), whose root is labeled by b and leaves are labeled by a 1 , . . . , a n , simply as b(a 1 . . . a n ). When a 1 = . . . = a n = a, write t = b(a n ). By a slight abuse of notation, for a unary tree t = a 1 (a 2 (. . . (a n ) . . .)), we write t = a 1 a 2 . . . a n for abbreviation. When a 1 = . . . = a n = a, we write t = a n for short. (In each case it should be clear from the context whether a n refers to a sequence of leaves or to a unary tree.) Next we briefly recall the definitions of the two variants of deterministic bottom-up tree automata considered here. A weakly deterministic unranked tree automaton (WDTA) is a 4-tuple A = (Q, Σ, δ , F) where Q is a finite set of states, Σ is the alphabet, F ⊆ Q is the set of final states, δ is a mapping from Q × Σ to the subsets of (Q ∪ Σ) * which satisfies the condition that, for each q ∈ Q, σ ∈ Σ, δ (q, σ ) is a regular language and for each label σ and every two states q 1 = q 2 , δ (q 1 , σ ) δ (q 2 , σ ) = / 0. The language δ (q, σ ) is called the horizontal language associated with q and σ and it is specified by a DFA H A q,σ . Roughly speaking, a WDTA operates as follows. If A has reached the children of a σ -labelled node u in states q 1 , q 2 ,..., q n , the computation assigns state q to node u provided that q 1 q 2 ...q n ∈ δ (q, σ ). In the sequence q 1 q 2 ...q n an element q i ∈ Σ is interpreted to correspond to a leaf labeled by that symbol. A WDTA is a deterministic hedge automaton [2] where each horizontal language is specified using a DFA.
Note that in the usual definition of [2] the horizontal languages are subsets of Q * . In order to simplify some constructions, we allow also the use of symbols of the alphabet Σ in the horizontal languages, where a symbol σ ∈ Σ occurring in a word of a horizontal language is always interpreted to label a leaf of the tree. The convention does not change the state complexity bounds in any significant way because we use small constant size alphabets and we can think that the tree automaton assigns to each leaf labeled by σ ∈ Σ a particular state that is not used anywhere else in the computation.
A strongly deterministic unranked tree automaton (SDTA) is a 4-tuple A = (Q, Σ, F, δ ), where Q, Σ, F are similarly defined as for WDTAs. For each a ∈ Σ, the horizontal languages δ (q, a), q ∈ Q, are defined by a single DFA augmented with an output function as follows.
is a DFA and λ a is a mapping S a → Q. For all q ∈ Q and a ∈ Σ, the horizontal language δ (q, a) is specified by D a as the set {w ∈ (Q ∪ Σ) * | λ a (γ * a (s 0 a , w)) = q}. Intuitively, when A has reached the children of a node u labelled by a in states q 1 , . . . , q m (an element q i ∈ Σ is interpreted as a label of a leaf node), the state at u is determined (via the function λ a ) by the state that the DFA D a reaches after reading the word q 1 · · · q m . More information on SDTA's can be found in [3].
Given a tree automaton A = (Q, Σ, F, δ ), the states in Q are called vertical states. The DFAs recognizing the horizontal languages are called horizontal DFAs and their states are called horizontal states. We define the (state) size of A, size(A), as a pair of integers [|Q|, n], where n is the sum of the sizes of all horizontal DFAs associated with A.

Union and intersection
We investigate the state complexity of union and intersection operations on unranked tree automata. The upper bounds on the numbers of vertical states are similar for SDTAs and WDTAs, however the upper bounds on the numbers of horizontal states differ between the two models.

Strongly deterministic tree automata
The following result gives the upper bounds and the lower bounds for the operations of union and intersection for SDTAs.

Theorem 3.1 For any two arbitrary SDTAs
3 For integers m, n ≥ 1 and relatively prime numbers k 1 , k 2 , . . . , k m , k m+1 , . . . , k m+n , there exists tree languages T 1 and T 2 such that T 1 and T 2 , respectively, can be recognized by SDTAs with m and n vertical states, ∏ m i=1 k i + O(m) and ∏ m+n i=1+m k i + O(n) horizontal states, and i any SDTA recognizing T 1 ∪ T 2 has at least (m + 1)(n + 1) − 1 vertical states and ∏ m+n i=1 k i horizontal states. ii any SDTA recognizing T 1 ∩ T 2 has at least mn vertical states and ∏ m+n i=1 k i horizontal states.
The upper bounds on vertical and horizontal states are obtained from product constructions, and Theorem 3.1 shows that for the operations of union and intersection on SDTAs the upper bounds are tight for vertical states and almost tight for horizontal states.

Weakly deterministic automata
In this section, the upper bounds on the numbers of vertical and horizontal states for the operations of union and intersection on WDTAs are investigated, and followed by matching lower bounds on the numbers of vertical states.

Lemma 3.1 Given two WDTAs
The language L(A 1 ) ∪ L(A 2 ) can be recognized by a WDTA B ∪ with The language L(A 1 ) ∩ L(A 2 ) can be recognized by a WDTA B ∩ with The theorem below shows that the upper bounds for the vertical states are tight.

Concatenation of strongly deterministic tree automata
We begin by giving a construction of an SDTA recognizing the concatenation of two tree languages recognized by given SDTAs.
or there exists c ∈ c 1 ∪C 2 such that c ∈ E 1 σ . µ is defined as below: For any input a ∈ Σ, µ((c 1 , (C 2 , x), c 2 ), a) = (γ 1 σ (c 1 , a), ( For any input (p 1 , Write the computation above in an abbreviated form as µ((c 1 , When compute P ′ 2 , add nothing to P ′ 2 if any γ i σ (c, α) is not defined. Let p lea f ∈ Q 1 denote the state assigned to the leaf in A 1 substituted by a tree in L(A 2 ). λ B σ is defined as: for any final state e = (c 1 , The state in B has three components (p 1 , P 2 , q). p 1 is used to keep track of A 1 's computation where no concatenation is done. p 1 is computed by the first component c 1 in the state of H B σ . P 2 traces the computation where the concatenation takes place. In a state (c 1 , (C 2 , x), c 2 ) of H B σ , x = 1 (or x = 0) records there is (or is not) a concatenation in the computation. The third component q keeps track of the computation of A 2 . When a final state is reached in A 2 , which means a concatenation might take place, an initial state p lea f is added to P 2 , which is achieved by the λ B σ function in B. According to the definition of λ B σ , when λ 2 σ (c 2 ) ∈ F 2 , p lea f is always in the second component of the state. Exclude the cases when λ 2 σ (c 2 ) ∈ F 2 , and p lea f is not in the second component of the state, and we do not require B be complete. B has ( For our lower bound construction we define tree languages consisting of trees where, roughly speaking, each branch belongs to the worst-case languages used for string concatenation in [16] and, furthermore, the minimal DFA reaches the same state at an arbitrary node u in computations starting from any As shown in Figure 1, A and B are the DFAs used in Theorem 1 of [16] except that a self-loop labeled by an additional symbol d is added to each state in B. We use the symbol d as an identifier of DFA B, which always leads to a dead state in the computations of A. This will be useful for establishing that all vertical states of the SDTA constructed as in Lemma 4.1 are needed to recognize the concatenation of tree languages defined below.
Based on the DFAs A and B we define the tree languages T A and T B used in our lower bound construction. The tree language T B consists of Σ-labeled trees t, Σ = {a, b, c, d}, where: 1. All leaves are labeled by a and if a node u has a child that is a leaf, then all the children of u are leaves.
2. B accepts the string of symbols labeling a path from any node of height one to the root.
3. The following holds for any u ∈ dom(t) and any nodes v 1 and v 2 of height one below u. If w i is the string of symbols labeling the path from v i to u, i = 1, 2, then B reaches the same state after reading strings w 1 and w 2 .
Intuitively, the above condition means that when, on a tree of T B , the DFA B reads strings of symbols labeling paths starting from nodes of height one upwards, the computations corresponding to different paths "agree" at each node. This property is used in the construction of an SDTA M B for T B below. Note that the computations of B above are started from the nodes of height one and they ignore the leaf symbols. This is done for technical reasons because in tree concatenation a leaf symbol is replaced by a tree, i.e., the original symbol labeling the leaf will not appear in the resulting tree.
where if p = m − 1 then 0 ∈ S, and if S = / 0 then q = n and p = m can not both be true. The number of states in (1) is (n + 1)((m + 1)2 n − 2 n−1 ) − 1. State q = n (or p = m) denotes q = dead (or p = dead) in the construction of lemma 4.1. We will show that C needs at least (n + 1)((m + 1)2 n − 2 n−1 ) − 1 vertical states. We prove this by showing that each state in (1) is reachable and all states are pairwise inequivalent, or distinguishable. Here distinguishability means that for any distinct states q 1 and q 2 there exists t ∈ T Σ [x] such that the (unique deterministic) computation of C on t(x ← q 1 ) leads to acceptance if and only if the computation of C on t(x ← q 2 ) does not lead to acceptance.

Lemma 4.2 All states of C are reachable.
Proof. We introduce the following notation. For a unary tree t = a 1 (a 2 (. . . a m (b) . . .)), we denote word(t) = a m a m−1 . . . a 1 ∈ Σ * . Note that word(t) consists of the sequence of labels of t from the node of height one to the root, and the label of the leaf is not included.
We show that all the states in (1) are reachable by using induction on |S|.
Now consider the case when y = m − 1. According to the definition of (1), 0 ∈ S ′ . According to the inductive assumption, state (x, S ′ − {0}, m − 2) is reachable. Then state (x, S ′ , m − 1) is reachable by reading a unary symbol a.
Since (x, S ′ , y) is an arbitrary state with |S ′ | = z + 1, we have proved that all the states (x, S ′ , y), Thus, all the states in (1) are reachable.  For any integers m, n ≥ 1, there exists tree languages T A and T B , such that T A and T B can be recognized by SDTAs having m and n vertical states, respectively, and any SDTA recognizing T A · T B needs at least (n + 1)((m + 1)2 n − 2 n−1 ) − 1 vertical states.
We do not have a matching lower bound for the number of horizontal states given by Lemma 4.1. With regards to the number of vertical states, both the upper bound of Lemma 4.1 and the lower bound of Theorem 4.1 can be immediately modified for WDTAs. (The proof holds almost word for word.) In the case of WDTAs, getting a good lower bound for the number of horizontal states would likely be very hard.

Conclusion
We have studied the operational state complexity of two variants of deterministic unranked tree automata. For union and intersection, tight upper bounds on the number of vertical states were established for both strongly and weakly deterministic automata. An almost tight upper bound on the number of horizontal states was obtained in the case of strongly deterministic unranked tree automata. For weakly deterministic automata, lower bounds on the numbers of horizontal states are hard to establish because there can be trade-offs between the numbers of vertical and horizontal states. This is indicated also by the fact that minimization of weakly deterministic unranked tree automata is intractable and the minimal automaton need not be unique [10].
As ordinary strings can be viewed as unary trees, it is easy to predict that the state complexity of a given operation for tree automata should be greater or equal to the state complexity of the corresponding operation on string languages. As our main result, we showed that for deterministic unranked tree automata, the state complexity of concatenation of an m state and an n state automaton is at most (n + 1)((m + 1)2 n − 2 n−1 ) − 1 and that this bound can be reached in the worst case. The bound is of a different order than the known state complexity m2 n − 2 n−1 of concatenation of regular string languages.