{'fraction_non_alphanumeric': 0.04231449081923452, 'fraction_numerical': 0.00645719851376213, 'mean_word_length': 4.189442156495694, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Abstract Model Theoretic Syntax and Parsing: An Application to Temporal Logic

In general, model-theoretic approaches to syntax aim to describe the syntactic structures of natural languages by means of logical formulae so that the structures are models of these formulae. However, in practice, the use of such descriptions for processing issues involves so much complexity, that they appear to be more or less useless for such tasks. We present a simple formalism, which is based on temporal logic, to specify sets of labeled binary trees. Then we show how to employ such temporal formulae in an Earley-style parsing formalism using labeled tableau methods. This may reduce the complexity, since the resulting parsing formalism only considers the models whose leaves represent the input string.


Introduction
Inspired by constraint-based and principle-based grammar formalisms, several model-theoretic approaches [8] to syntax have attracted attention over the last decade. Insofar as they differ from one another, they do so in their underlying logical framework; for instance Kracht [19] employs propositional dynamic logic, Palm [21,22] temporal logic and a fragment of propositional dynamic logic, Blackburn et al. [3] modal logic, Rogers [24] weak monadic second order logic. Such a formalism considers syntactic structures to be the models of a particular formula specifying the syntactic properties of a language. Typically, these structures are ordered trees whose nodes are labeled with symbols or propositions of a finite domain.
Although these formalisms tell us a lot about the properties and the nature of such trees, we do not hear much about operational aspects like parsing or generation. In the approach presented here, we introduce a parsing method Palm that employs a logical description of trees as its grammatical base. The underlying logical framework is a particular version of propositional branching time logic, since this formalism already provides tree-shaped models. Although the generative power of temporal logic lies somewhere between dynamic and modal logic, it should be sufficient for most linguistic applications to stay within the scope of context-free languages. On the basis of this formalism, we construct a tableau calculus that builds valid tree models for a given formula. Afterwards, we present an Earley-based parsing formalism for temporal formulae that employs this tableau calculus to dynamically generate the grammatical information which is necessary for parsing an input string.

A Propositional Temporal Logic for Binary Trees
According to its basic nature, branching temporal logic provides a canonical method to describe certain sets of trees. In such a framework the notion of time is organized into discrete points of time where each of them owns alternative possible successors. If we further assume that branching time lines never join together and that there is a unique starting point, we obviously obtain tree-like structures. As a common application, such trees are considered to be the alternative computations of finite-state concurrent systems, leading to computation tree logic CTL [5,6] and related formalisms. However, we must take into account the basic difference between CTL structures and the trees considered here: the number of tree nodes is finite and there is a linear order imposed on the children of a node. Further, we restrict our attention to binary trees, since binary constituent structures typically occur in natural languages. Thus, they are commonly used by natural language grammar formalisms e.g. Head-driven Phrase Structure Grammar (HPSG), Government and Binding (GB), Minimalist Program (MP), Categorial Grammar (CG).
Looked at in formal terms CTL and related formalisms follow Arthur Prior's [23] view, where temporal logic is used as a particular version of modal logic which is more powerful than ordinary modal logic but which is also properly embedded in propositional dynamic logic. Moreover, most versions of computation tree logic distinguish between paths and nodes, which is essential for specifying certain properties of (infinitely running) concurrent systems. In the framework considered here, the situation is less complex, since due to the finiteness we could show that each leaf in a finite tree uniquely represents a path and, hence, a path might be viewed as a leaf with particular properties. Hence the conditions on path can be transformed into conditions on tree nodes. As a consequence the syntax used for our formalism resembles more propositional linear time logic, despite of the fact that we actually describe branching structures.
Given this view of trees, we only require the modalities that access to the next immediate temporal states and the until-operator. For a binary tree, we need two next-operators accessing the next left and the next right state.

Palm
Correspondingly, our language of propositional binary branching temporal logic PBTL 2 employs three basic modalities: the two operators • 1 and • 2 , which access the first (left) and second (right) successor, and the until-operator U that access to a sequence of nodes until a certain condition becomes true. Definition 2.1 (Syntax of PBTL 2 ) Given a finite set of propositions P = {p, q, r, . . .}, the set of PBTL 2 formulae (over P) is defined as follows: Based on these operators, we can define the Boolean connectives ∨, →, ↔ in the usual way and, in addition, the temporal operators next •ϕ ≡ As mentioned above the underlying structures of PBTL 2 consist of ordered finite binary trees, where each node is labeled with a set of propositions of a given set P. In contrast to standard temporal logic, we assume that every branch of the tree ends at a leaf node and includes at most a finite number of nodes. We represent such a structure as a finite subset t ⊂ {1, 2} * which is closed under prefixes, i.e. for all w.a ∈ t with w ∈ {1, 2} * and a ∈ {1, 2} we have w ∈ t, where ε denotes the root. In addition, a right successor always requires a corresponding left successor, i.e. for all w.2 ∈ t there is w.1 ∈ t. We interpret PBTL 2 formulae over triples (t, h, n) representing nodes n ∈ t in a labeled binary tree domain T = (t, h) where t ⊆ {1, 2} * is a tree domain and h : t → 2 P is a labeling function: • (t, h, n) |= and (t, h, n) |= ⊥.
The tree T = (t, h) satisfies a PBTL 2 formula ϕ if and only if the root of T satisfies ϕ, i.e. (t, h, ε) |= ϕ. As an alternative to this root-based semantic of (t, n) |= ϕ, we could employ the universal interpretation, where all nodes n of (t, h) must satisfy ϕ but which would be equivalent to (t, h) |= 2ϕ. For our purpose we prefer the root-based semantic since this interpretation enables us to construct trees starting from the root, while separating and analyzing a formula.
Another aspect we have not taken into account so far concerns the expressive power of PBTL 2 . In temporal terms, this formalism only considers the 'future' of a state but ignores its past. Nevertheless, there are some re-sults concerning the strong correspondence that holds between temporal logic, past temporal logic and first-order logic. For the linear case this can be obtained from [18,14] and for binary trees from [17]. If this were applied to PBTL 2 , additional past operators would not increase the overall expressivity. For instance, if we want to state that all nodes satisfying some condition ϕ must have a parent satisfying ψ, we could employ the non-past formulation ¬ϕ ∧ 2ϕ(•ϕ → ψ). Accordingly, the models of PBTL 2 are the set of binary trees described by a corresponding version of first-order logic on binary trees.

A Tableau-based Construction of Tree Models
Now we introduce a method that generates the tree models of a given PBTL 2 formula. Our solution follows similar approaches to constructing corresponding tree automata [10,2,27]. Unfortunately, these approaches employ variants of alternating tree automata for infinite trees, which allow linear time model checking, but require in general exponential time for generating at least one tree. Therefore, we employ a tableau-based method that constructs plain tree models which relates to similar methods in the CTL framework [1,11,13,28,20]. Typically, a tableau calculus denotes a proof method, however, it can also be viewed as a formalism that constructs models for a given formula. Within a modal framework we can extend the tableau formulae with certain labels that indicate the underlying reachability relation. Consequently, such a labeled tableau system may provide the models of a given formula. The construction fails if all branches of the tableau are closed. But every branch that is not closed represents a model of the underlying formula where the labels of the formulae occurring in this branch represent the internal structure of that model. Afterwards, we use this way of constructing tree models to handle the parsing actions of a modified Earley-parser for PBTL 2 .
In formal terms, a tableau calculus denotes a system of rules that extends a given set of formulae in a particular manner. The result is a tree-like structure, where each branch denotes a certain alternative. For a detailed discussion of tableau methods in a modal or a temporal framework, we refer to [4,15]. A tableau rule with the name (α) consists of a premise P and a finite number of alternative conclusions C 1 | . . . |C k written as (α) P C 1 | · · · |C k In our approach, the premise consists of a single labeled formula, while the conclusions are sets of labeled formulae. We read a tableau rule downward, i.e. "if the premise is satisfiable then so is at least one of the conclusions". A finite set of tableau rules, which are identified by their rule names, denotes a tableau calculus T C. Within our approach, a given non-atomic formula matches exactly the premise of a single tableau rule. Therefore no other tableau rule can be applied to this formula. For a given finite set of formulae X = {ϕ 1 , . . . , ϕ k } we construct the tableau tree T C(X) according to the Palm following steps: • Each tree node represents a single formula, where we arrange the initial formulae ε:: ϕ 1 , . . . , ε:: ϕ k in an arbitrary vertical linear sequence and mark them as active nodes.
• Atomic formulae σ:: p and σ:: ¬p for some p ∈ P are always inactive and can never be activated.
• If an active formula σ:: ϕ in an open branch B matches the premise of a rule, then we add its conclusions as active nodes at the end of the branch so that a rule with n conclusions leads to an n branching node. We only add labeled formulae that do not already occur on this branch, except this would rule out a certain alternative. In this case we still add this formula but mark it as inactive [16].
• After we handled all open branches of the active formula under consideration, this formula becomes inactive.
• If we add σ:: ⊥ to any branch or if we add σ:: ¬ψ to some branch including σ:: ψ, then the branch under consideration is marked as closed. A node also becomes inactive if all branches including it are closed.
A tableau is closed if all branches are closed. A branch is finished if either it is closed or no rule can be applied to one of its formulae (without introducing new formulae on this branch). The tableau is finished if all branches are finished. To express the immediate dominance relation of the tree models, we associate each formula with a particular label σ where the labels σ.1 and σ.2 indicate the left and the right successors of a node. These labels distinguish formulae that are assigned to different tree nodes and they express how the tableau formulae are structurally related to each other.
To construct the tableau rules, we reconsider the (intended) semantics of the PBTL 2 formulae. Obviously, the rules should handle each type of formula uniquely by separating it into corresponding subformulae obeying the particular structure of binary trees and the formal properties of temporal and modal logic. For each operator, except the one for negation, we consider the original positive version and its negated counterpart. Thus, we must handle the negation operators only for the propositions in P, the Boolean constants and ⊥ and the atomic next-operators • 1 and • 2 : Obviously, these rules state again the semantic properties of PBTL 2 . The auxiliary propositions leaf 1 and leaf 2 handle the atomic next-operators • 1 , • 2 , ¬• 1 and ¬• 2 , which indicate whether a node has no left or no right successor, respectively. For pure propositional formulae, the structure below a node does not affect its validity in any way. Therefore, a simple tableau with the single node ε:: p represents all binary trees, where the root includes the proposition p. Accordingly, for some open branch B in a tableau T C(X) we define T (B) as the set of binary trees described by B that properly meet the auxiliary propositions leaf 1 and leaf 2 , too. In addition, the set T T C (X) denotes the set of binary trees described by the tableau T C(X). An issue we ignored so far concerns the finiteness of the trees considered. Actually, the tableau rules presented above do not force closing the tableau for PBTL 2 formulae that only have infinite models. For instance the formula 2•p (for some p ∈ P) only holds for trees where every node has at least one successor satisfying p. Consequently, it fails for finite trees but the corresponding T C-tableau does not close. In order to prevent this we employ Löb's axiom ( φ → φ) → φ where denotes the weak version of 2 referring to all descendants of a node but not to the node itself (i.e. φ ≡ ¬•¬2φ) as done in [3] for a similar approach. 2 As shown by Smorynski [26] this axiom ensures the absence of infinite sequences of immediately dominating tree nodes n, na 1 , na 1 a 2 , . . . for n ∈ {0, 1} * and a i ∈ {0, 1}. After transforming this axiom to 3φ → 3(φ ∧ ¬3φ) we obtain that if there is a node satisfying some formula ϕ then there must be a node satisfying φ but its descendants fail φ. Moreover a formalism fails this axiom if it includes 3φ ∧ (φ → 3φ). Considering the tableau rules one realizes that only the (¬ U ) rule can pass a formula from a node to all descendants. Obviously, the only possibility to violate Löb's axiom arises if for σ:: ¬(ψ Uϕ) all branches not including σ.1:: ¬(ψ Uϕ) and σ.2:: ¬(ψ Uϕ) are always closed. This is only true if ¬ψ, ¬ϕ and ψ, ¬ϕ, leaf 1 , leaf 2 are inconsistent. Correspondingly, we add a further closing condition to our tableau formalism which ensures that the branch including σ:: ¬(ψ Uϕ) closes in this case. Theorem 3.1 T C is complete, i.e. for every finite binary tree (t, h) and every PBTL 2 formula ϕ it is true that: Proof (Sketch) By the way of constructing the rules presented above, we obtain, more or less obviously, the conclusion that every finite binary tree (t, h) ∈ T T C (ϕ) satisfies the formula ϕ. However, we need more effort to establish the opposite direction, claiming that every finite binary tree satisfying some PBTL 2 formula ϕ can be obtained by this tableau calculus. We must show that for every finite binary tree T = (t, h) satisfying ϕ there is an open branch B T ∈ T C(ϕ) such that B T represents (t, h). By induction on the tree structure we show that we can construct an open branch B T , where all nodes n in T meet the following conditions: (i) If p ∈ h(n) then n:: ¬p ∈ B T .
(ii) If p ∈ h(n) then n:: p ∈ B T .
(iii) If n is not in B T then there is an ancestor n p such that n p is an Ω-node, i.e. n p :: a ∈ B T for any a ∈ {leaf 1 , ¬leaf 1 , leaf 2 , ¬leaf 2 } The last condition indicates the case where the structure below n p is arbitrary. For the induction we assume an unfinished tableau where we extend B in an appropriate manner by employing tableau rules. We start the induction by considering the case (t, h, ε) |= ϕ. Since ϕ has a model and our tableau calculus is correct, T C(ϕ) includes at least one open branch. Due to the initialization, every branch of the tableau T C(ϕ) starts with ε:: ϕ. Thus, as demanded, there is at least one open branch including ε:: ϕ. Now we consider some nodes n = a 1 . . . a i+1 in T with a 1 , . . . , a i+1 ∈ {1, 2}. If n p is a valid label in B, we show that we can expand B by employing individual tableau rules so that it includes the appropriate formulae wearing the label n. Otherwise, if n p is absent in B, it must meet the condition (iii) and so does n and any other descendant of n p . Now we show how to determine the appropriate n-labeled formulae, but only for the case, where n = n p .1 and n p :: ρ Uψ. The proof for n = n p .2 and the remaining formulae is similar. From the tableau construction follows that ε:: ϕ entails n p :: ρ Uψ and consequently (t, h, n p ) |= ρ Uψ. Now we can expand B in three alternative ways using the U-rule. The first alternative n p :: ψ makes no statements concerning n. Thus, if no other formula adds a (negated) auxiliary label, n p is an Ω-node. The second one adds n:: ρ Uψ and the third one implicitly adds n:: . According to the semantics of PBTL 2 as specified in Def. 2.2, n p must satisfy ψ or it must have a successor n p .a with a ∈ {1, 2} that satisfies ρ and ρ Uψ. If the first one is true, we select the first alternative. Otherwise we select the second or the third alternative, if a = 1 or a = 2, respectively. Afterwards it could be necessary to employ some propositional rules (∧), (∨), (¬). However their application would never close the branch B. Consequently, the node n meets the conditions we claimed above for the resulting expanded branch. 2 4 An Earley-Parser for PBTL 2 The tableau formalism introduced above can be employed to construct all binary trees that are models for a given PBTL 2 formula. It is further possible to create a tree automaton generating this set of trees, where the states q ∈ Q correspond to sets of formulae wearing the same label and where the transition relation δ ⊆ Q × 2 P × Q × Q can be obtained as follows: for some tableau branch B and some label σ where σ(B) := {ϕ | σ:: ϕ ∈ B} and S ∪ σ(B) ⊥ expresses that the propositions p ∈ S are consistent with σ(B). However, the number of states might be exponential with respect to the size of the underlying PBTL 2 formulae, and so might be the resulting tree automaton.
Although when a tree automaton and, hence, the corresponding context-free grammar is used, parsing an input sentence requires, at most, cubic time with respect to the length of the input, this time could be exponential with respect to the length of the underlying PBTL 2 formula. We try to avoid this exponential growth by selecting only those parts of the transitions that are necessary for parsing a given input sentence. Nevertheless, in general, exponential growth cannot be avoided. Such cases occur if the input imposes only a weak restriction on the possible structures. For instance, the formula 2(p 1 ∨ . . . ∨ p k ) with P = {p 1 , . . . , p k } accepts any input string, but it has k alternatives for each node. Since PBTL 2 works downward in a tree, we employ a top-down parsing formalism, namely an Earley-parser [9]. Instead of employing the rules of a context-free grammar, we dynamically generate triples of sets of consistent PBTL 2 formulae using the tableau formalism defined above. For a given PBTL 2 formula ϕ we assume to have the whole finished tableau T C(X), but actually we only dynamically generate the required parts of it. Now let A be a set of all PBTL 2 formulae wearing the same label σ in some branch of T C(X), where X = {ϕ 1 , . . . , ϕ m } is the given finite set of PBTL 2 formulae. Then, by applying the T C-rules to A, we can construct alternative pairs of sets B and C, representing the left and right successor of A. Provided that A is T C-saturated, i.e. no more T C-rules can be applied to it, we do not need any information concerning the branch or the label σ, and the sets B and C are T C-saturated, too. For an absent successor, the corresponding set B or C is empty where the auxiliary propositions leaf 1 and leaf 2 must be set accordingly. If A denotes a proposition, i.e. no temporal operator occurs and, hence, no auxiliary proposition, the structure below it is arbitrary, which is expressed by the special symbol Ω; thus, we set B = C = Ω. Hence, if A, B and C meet these conditions, we write A, B, C ∈ T C(X). Moreover, Palm root(T C(X)) denotes the set of initial sets which are the T C-saturated sets of formulae wearing the label ε.
Basically, an Earley-type parser works on a set of parsing items. Within our approach a parsing item p, A, B, C, i, j states that we have recognized the part w i+1 . . . w j of the input string w = w 1 . . . w n (with 0 ≤ i ≤ j ≤ n) for a subtree with the root A and the left and right successor B and C, respectively. The position p ∈ {l, m, r} indicates the position of the 'dot': p = l, the left position, i.e. no part of the input string was recognized; p = m, the middle, i.e. the part below B has been recognized, but the part below C still needs to be recognized; p = r, i.e. the right position, the whole subtree has been recognized successfully. The set of parsing items I for an input string w 1 . . . w n is built by the following Earley-parsing operations: The operation Initialize introduces the root properties to the item set. Next we recursively employ the expand operation to predict the possible left descendants of the root. After there is no further possible prediction, we read the first input symbol using the shift operation. A successful shift moves the 'dot' from the left to the right position in the corresponding item. Moreover, every shift possibly triggers a complete operation which completes an item predicted previously. In addition, this can also trigger further complete operations. Likewise, we repeat this scheme of expand, shift and complete operations until the whole input string has been proceeded. We accept the input string if I includes the item r, A, B, C, 0, n for some A, B, C ∈ root(T C(X)). Actually, each item r, A, B, C, i, j ∈ I describes a set of (sub-)trees T I (A, i, j) over w i+1 . . . w j , where the root satisfies A. This set can be obtained as follows:  (A, 0, n) is the set of parse trees for the input string w 1 · · · w n . Our parsing method combines tableau resolution with a standard Earleyparser using forward chaining techniques. To establish the intended behaviour we must show that the parser is correct, i.e. every resulting parse tree (t, h) satisfies X, and more important, the parser is complete, i.e. it generates every tree that matches the input string and that satisfies the underlying tree formulae.

Theorem 4.1 The Earley-based parser for PBTL 2 is correct and complete.
Proof (Sketch) Since the expand operation employs the tableau rules, it only predicts parts of the tree that occur in the corresponding branch of the tableau T C(X). However, each parse tree consists of completed items that were predicted previously, so it must be in T T C (X) and, according to Theorem 3.1, it satisfies ϕ. For the completeness we employ the method presented by Sikkel [25] that shows by induction that the parser can generate every item p, A, B, C, i, j that corresponds to tree node n. In our case this node means some label σ with A = σ(B), B = σ.1(B), C = σ(B) for some branch B ∈ T C(X) where the leaves of B match the input string w. Basically, we must determine the minimal number d( p, A, B, C, i, j ) of parsing operations that are necessary to generate this item. Then, by induction on this number, any item generated by k minimal operations is generated by items requiring less than k operations. To employ this method we must carry out some modifications. One concerns the number of context-free derivations, the second one concerns the use of context-free rules, since either of these issues does not directly match our formalism. Obviously, the number of derivation steps s in a context-free derivation A ⇒ s w i . . . w j is equal to the number of nonleaf nodes in the corresponding subtree. Moreover we can consider any triple A, B, C ∈ T C(ϕ) as a context-free rule A → BC. When using these modifications, we can apply the proof of completeness presented in [25] to our approach. 2 Finally, we make some remarks concerning the complexity of our parsing formalism for PBTL 2 formulae. Obviously, the Earley-based part requires at most cubic time with respect to the length of the input string. However, the crucial value results from the number of triples A, B, C ∈ T C(X) provided by the tableau. Their number is at most exponential with respect to the length of the given formula, since the sets A, B, C correspond to subsets of the set of subformulae obtained from the given formula. Once again we consider the tableau as a tree-automaton, that includes an exponential number of states q A in the worst case. Thus it is possible to consider an exponential number of triples A, B, C while parsing an input string. Now we can interpret the resulting set of parsing items as the result obtained by intersecting the tree automaton for the grammar and the tree automaton generating all trees over the given input string w, which has been already suggested in [7]. For the second tree automaton, we only require states for each pair (i, j) with 0 ≤ i < j ≤ n so that a state q i,j spans over the section w i+1 . . . w j of the input string. The number of such states is bound by n 2 . After intersecting both tree automata, we obtain states consisting of pairs of states q A and q i,j , which correspond to the parsing items. However, the exponential worst case scenario remains. But this may not apply to every input string and every underlying PBTL 2 formula ϕ. For instance, if ϕ includes strong structural restrictions that only allow a small number of structures over (a part of) the input string, then only a small section of the states must be considered. In general, we can approximately state that the more structural restrictions we have, the less is the complexity and the less is the number of possible valid trees for some input string. However, in natural language we commonly obtain only a small number of possible syntactic structures for an input sentence. Thus one could expect a complexity far below exponential for such applications.

Concluding Remarks
We have sketched an Earley-like parsing formalism that is based on a modeltheoretic description of syntax by means of a particular version of branching time logic. Although this approach makes use of the strong relationship between branching time and top-down parsing, this is not a general limitation. Obviously, one might capture other methods like head-corner parsing by introducing appropriate temporal modalities.
For evaluation purposes, some parts of this approach have been implemented in Prolog. However, the results we have obtained so far are too weak to make any general statements in this respect. Nevertheless, the simplicity of the Earley-formalism presented above offers several points of departure for improving time and space complexity. Another area of improvement would involve combing it with (partially) lexicalized grammars, where lexical structures are given as additional formulae that can be included in the expand operation, which must be adopted correspondingly.
Another way improving the complexity may result from restricting the syntax of PBTL 2 as proposed in [12] for the case of CTL. However such a restriction includes a maximal number of nested temporal modalities. Therefore we must check whether such a constraint also applies to the syntax of natural languages. Nevertheless this is in general true for the examples proposed in