{'fraction_non_alphanumeric': 0.039024235099548374, 'fraction_numerical': 0.011099802811525984, 'mean_word_length': 4.30325518637207, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sparse Nested Markov models with Log-linear Parameters

Hidden variables are ubiquitous in practical data analysis, and therefore modeling marginal densities and doing inference with the resulting models is an important problem in statistics, machine learning, and causal inference. Recently, a new type of graphical model, called the nested Markov model, was developed which captures equality constraints found in marginals of directed acyclic graph (DAG) models. Some of these constraints, such as the so called `Verma constraint', strictly generalize conditional independence. To make modeling and inference with nested Markov models practical, it is necessary to limit the number of parameters in the model, while still correctly capturing the constraints in the marginal of a DAG model. Placing such limits is similar in spirit to sparsity methods for undirected graphical models, and regression models. In this paper, we give a log-linear parameterization which allows sparse modeling with nested Markov models. We illustrate the advantages of this parameterization with a simulation study.


Introduction
Analysis of complex multidimensional data is often made difficult by the twin problems of hidden variables, and a dearth of data relative to the dimension of the model. The former problem motivates the study of marginal and/or latent models, while the latter has resulted in the development of sparsity methods. A particularly appealing model for multidimensional data analysis is the Bayesian network or directed acyclic graph (DAG) model [10], where random variables are represented as vertices in the graph, with directed edges (arrows) between them. The popularity of DAG models stems from their well understood theory, and from the fact that they elicit an intuitive causal interpretation: an arrow from a variable A to a variable B in a DAG model can be interpreted, in a way which can be made precise, to mean that A is a 'direct cause' of B.
DAG models assume all variables are observed, and a latent variable model based on DAGs simply relaxes this assumption. However, latent variables introduce a number of problems: it is difficult to correctly model the latent state, and the resulting marginal densities are quite challenging to work with. An alternative is to encode constraints found in marginals of DAG models directly; a recent approach in this spirit is the nested Markov model [15]. The advantage of the nested Markov model is that it correctly captures the conditional independences and other equality constraints found in marginals of DAG models. However, the discrete parameterization of nested Markov models has the disadvantage of being unable to represent constraints in various marginals of DAGs concisely, that is with few non-zero parameters. This implies that model selection methods based on scoring (via the BIC score [13] for instance) often prefer simpler models which fail to capture independences correctly, but which contain many fewer parameters [15].
More generally, in high dimensional data analyses there is often such a shortage of samples that classical statistical inference techniques do not work. To address these issues, sparsity methods have been developed, which drive as many parameters in the statistical model to zero as possible, while still providing a reasonable fit to the data. Sparsity methods have been developed for regression models [16], undirected graphical models [8,9], and even some marginal models [4].
It is not natural to apply sparsity techniques to existing parameterizations of nested Markov models, because the parameters are context (or strata) specific. In this paper, we develop a log-linear parameterization for discrete nested Markov models, where the parameters represent (generalizations of) log odds-ratios within 'kernels' (informally 'interventional' densities). These can be viewed as interaction parameters, of the kind commonly set to zero by sparsity methods. Our parameterization allows us to represent distributions containing 'Verma constraints' in a sparse way, while maintaining advantages of nested Markov models, and avoiding the disadvantages of using marginals of DAG models directly.

Disadvantages of the Möbius Parameterization of Nested Markov Models
One drawback of the standard parameterization of nested Markov models is that parameters are variation dependent; that is, fixing the value of one parameter constrains the 'legal' values of other parameters. This is in direct contrast with parameterizations of DAG models where parameters associated with a particular Markov factor (a conditional density for a variable given all its parents in the DAG) do not depend on parameters associated with other Markov factors.
We illustrate another difficulty with an example. Here, and in subsequent discussions, we will need to draw distinctions between vertices in graphs, and corresponding random variables in distributions or 'kernels.' We use the following notation: v (lowercase) denotes a vertex, X v the corresponding random variable, and x v a value assignment to this variable. Likewise A (uppercase) denotes a vertex set, X A the corresponding random variable set, and x A an assignment to this set.
Consider the marginal DAG shown in Fig. 1 (a). We wish to avoid representing this domain with a DAG directly, in order not to commit to a particular state space of the unobserved variables X 6 and X 7 , and because, even if we were willing to make such an assumption, the margin over (X 1 , X 2 , X 3 , X 4 , X 5 ) obtained from a density that factorizes according to this DAG can be complicated to work with [7].
To use nested Markov models for this domain, we first construct an acyclic directed mixed graph (ADMG) that represents this DAG marginal, using the latent projection algorithm [17]. This graph is shown in Fig.  1  If we now use the nested Möbius parameters, described in more detail in subsequent sections, to parameterize the resulting ADMG, we will quickly discover that this results in a model of higher dimension relative to the dimension of DAG models which share their skeleton with this ADMG. For example, the binary nested Markov model of the graph in Fig. 1 (b) has 16 parameters, while both binary DAG models corresponding to graphs in Fig. 7 (a) and (b) have 11 parameters each.
This leads to a worry that a structure learning algorithm that tries to use nested Möbius parameters to recover an ADMG from data by means of a score method, such as BIC [13], which rewards fit and parameter parsimony, may prefer at low sample sizes incorrect independence models given by DAGs in preference to correct models given by ADMGs, simply because the DAG models compensate for their poor fit of the data with a much smaller parameter count. In fact, this precise issue has been observed in simulation studies reported in [15].
Addressing this problem with a Möbius parameterization is not easy, because Möbius parameters are strata or context-specific; in other words, the parameterization is not independent of how the states are labeled. For instance, some of the Möbius parameters representing confounding between X 2 ,X 4 and X 5 are: 1 for all values of x 1 , x 3 . In a binary model, this gives 4 parameters. The kinds of regularities in the true generative process, which we may want to exploit to create a dimension reduction in our model, typically involve a lack of interactions among variables, or a latent confounder with a low dimensional state space. Such regularities may often not translate into constraints naturally expressible in terms of Möbius parameters.
To avoid this difficulty, we need to construct parameters for nested Markov models which represent various types of interactions among variables directly. In fact, parameters representing interactions are well known in log-linear models, of which undirected graphical models and certain regression models form a special case.

Log-linear Parameters for Undirected Models
We will use undirected graphical models, also known as Markov random fields, to illustrate log-linear models. A Markov random field over a multivariate binary state space X V , is a set of densities p(x V ) represented by an undirected graph G with vertices V , where here cl(G) is the collection of (not necessarily maximal) cliques in the undirected graph, · 1 is the L 1 -norm, and λ C is a log-linear parameter. Note that the parameter λ ∅ ensures the expression is normalized.
Consider the undirected graph shown in Fig. 2. In this graph, all subsets of {1, 2, 3}, {2, 4}, and {4, 5, 6} are cliques. The model represents densities where, conditional upon its adjacent nodes, each node is independent of all others. The log-linear parameter(s) corresponding to each such subset of size k can be viewed as representing k-way interactions among appropriate variables in the model. Setting some such interaction parameters to zero in a consistent way results in a model which still asserts the same conditional independences, but has a smaller parameter count, and with all strata in each clique treated symmetrically. For instance, if we were to set all parameters for cliques of size k > 2 to zero, so that there remained only parameters corresponding to vertices and individual edges, we would obtain a model known as a Boltzmann machine [1]. A similar idea had been used to give a sparse parameterization for discrete DAG models [12].
In the remainder of the paper, we describe nested Markov models, and give a log-linear parameterization for these models which contains similar parameters that may be set to zero. While in Markov random field models the parameters are associated with sets of nodes which form cliques in the corresponding undirected graph, in nested Markov models parameters will be associated with special sets of nodes in the corresponding ADMG called intrinsic sets. Further, log-linear parameterizations of this type can often incorporate individual-level continuous baseline covariates [5].

Graphs, Kernels, and Nested Markov Models
We now introduce the relevant background needed to define the nested Markov model.
is a graph with a set of vertices V and a set of edges E, where the edges may be directed (→) or bidirected (↔). A directed cycle is a path of the form x → · · · → y along with an edge y → x. An acyclic directed mixed graph (ADMG) is a mixed graph containing no directed cycles. An example is given in Fig. 1 Let a, b and d be vertices in a mixed graph G. If b → a then we say that b is a parent of a, and a is a child of b. If a ↔ b then a is said to be a spouse of b. A vertex a is said to be an ancestor of a vertex d if either there is a directed path a → · · · → d from a to d, or a = d; similarly d is said to be a descendant of a. The sets of parents, children, spouses, ancestors and descendants of a in G are written pa G (a), ch G (a), sp G (a), an G (a), de G (a) respectively. We apply these definitions disjunctively to sets, e.g. an G (A) = a∈A an G (a).

Conditional ADMGs
A conditional acyclic directed mixed graph (CADMG) The vertices in V are the random vertices, and those in W are called fixed.
Whereas an ADMG with vertex set V represents a joint density p(x V ), a conditional ADMG represents the Markov structure of a conditional density, or kernel q V (x V |x W ). Following [8, p.46], we define a kernel to be a non-negative function q V (x V |x W ) satisfying: We use the term 'kernel' and write q V (·|·) (rather than p(·|·)) to emphasize that these functions, though they satisfy (1) and thus most properties of conditional densities, are not in general formed via the usual operation of conditioning on the event X W = x W . To conform with standard notation for densities, for every For a CADMG G(V, W, E) we consider collections of random variables (X v ) v∈V indexed by variables (X w ) w∈W ; throughout this paper the random variables take values in finite discrete sets ( That we will always hold the variables in W fixed is precisely why we do not permit edges between vertices in W . An ADMG G(V, E) may be seen as a CADMG in which W = ∅. In this manner, though we will state subsequent definitions for CADMGs, they also apply to ADMGs.
In forming G A , the status of the vertices in A with regard to whether they are in V or W is preserved.

Districts and Markov Blankets
A set C is connected in G if every pair of vertices in C is joined by a path such that every vertex on the path is in C. For a given CADMG G(V, W, E), denote by (G) ↔ the CADMG formed by removing all directed edges from G. A set connected in (G) ↔ is called bidirected connected.
For a vertex x ∈ V , the district (or c-component) of x, denoted by dis G (x), is the maximal bidirected connected set containing x. For instance in the ADMG shown in Fig. 1 (b), the district of node 2 is {2, 4, 5}. Districts in a CADMG form a partition of V ; vertices in W are excluded by definition. In a DAG G(V, E) the set of districts is the set of all single element sets

The fixing operation and fixable vertices
We now introduce a 'fixing' operation on a CADMG which has the effect of transforming a random vertex into a fixed vertex, thereby changing the graph. However, this operation is only applicable to a subset of the vertices, which we term the (potentially) fixable vertices.
is both a descendant of v and in the same district as v. For the graph in Fig. 1 (b), the vertex 2 is not fixable, because 4 is both its descendant and in the same district; all the other vertices are fixable.
Definition 2 Given a CADMG G(V, W, E), and a kernel q V (X V | X W ), with every r ∈ F(G) we associate a fixing transformation φ r on the pair (G, q V (X V | X W )) defined as follows: where E r is the subset of edges in E that do not have arrowheads into r, and Returning to the ADMG in Fig. 1 (b), fixing 3 in the graph means removing the edge 2 → 3, while fixing The resulting CADMG, shown in Fig. 3 (a), represents the resulting kernel q 1,2,4,5 (x 1 , x 2 , x 4 , x 5 |x 3 ).
We use • to indicate composition of operations in the natural way, so that: φ r • φ s (G) ≡ φ r (φ s (G)) and

Reachable and Intrinsic Sets
In order to define the nested Markov model, we will need to define special classes of vertex sets in ADMGs.
Fixing operations do not in general commute, and thus only some orderings are valid for fixing a particular set. For example, in the ADMG shown in Fig. 1 (b), the set {2, 3} may be fixed, but only under the ordering where 3 is fixed first, to yield the CADMG shown in Fig. 3 (a), and then 2 is fixed in this CADMG. Fixing 2 first in Fig. 1 (b) is not valid, because 4 is both a descendant of 2 and in the same district as 2 in that graph, and thus 2 is not fixable.
we say that the set V is reachable in G * . A reachable set may be obtained by fixing vertices using more than one valid sequence. We will denote any valid composition of fixing operations that fixes a set A by φ A if applied to the graph, and by φ X A if applied to a kernel. With a slight abuse of notation (though justified as we will later see) we suppress the precise fixing sequence chosen.
Definition 4 A set of vertices S is intrinsic in G if it is a district in a reachable subgraph of G. The set of intrinsic sets in an ADMG G is denoted by I(G).
In any DAG G(V, E), I(G) = {{x}|x ∈ V }, while in any bidirected graph G, I(G) is equal to the set of all connected sets in G.

Nested Markov Models
Just as for DAG models, nested Markov models may be defined via one of several equivalent Markov properties. These properties are all nested in the sense that they apply recursively to either reachable or intrinsic sets derived from an ADMG. In particular, there is a nested analogue of the global Markov property for DAGs (d-separation), the local Markov property for DAGs (which asserts that variables are independent of non-descendants given parents), and the moralizationbased property for DAGs. These definitions appear and are proven equivalent in [11]. It is possible to associate a unique ADMG with a particular marginal DAG model, and a nested Markov model associated with this ADMG will recover all independences which hold in the marginal DAG [11].
We now define a nested factorization on probability distributions represented by ADMGs using special sets of nodes called 'recursive heads' and 'tails.' Definition 5 For an intrinsic set S ∈ I(G) of a CADMG G, define the recursive head (rh) as: rh(S) ≡ {x ∈ S | ch G (x) ∩ S = ∅}.

Definition 6
The tail associated with a recursive head H of an intrinsic set S in a CADMG G is given by: In the graph in Fig. 1 (b), the recursive head of the intrinsic set {2, 4, 5} is equal to the set itself, while the tail is {1, 3}.
where V G is a partition of V into heads given in [14].
That is, G(G) is the set of valid fixing sequences and the CADMGs that they reach. The same graph may be reached by more than one sequence w * . We say that a distribution p(x V ) obeys the nested head factorization property for G if for all (G * , w * ) ∈ G(G), the kernel φ w * (p(X V ); G) obeys the head factorization for φ w * (G) ≡ G * . We denote the set of such distributions by P n h (G). Nested Markov models have been defined via a nested district factorization criterion [15], and a number of Markov properties [11]. The head factorization is another way of defining the nested Markov model due to the following result.

Theorem 7 The set P n h (G) is the nested Markov model of G.
Our decision to suppress the precise fixing sequence from the fixing operation applied to sets is justified, due to the following result.
Theorem 8 If p(x V ) is in the nested Markov model of G, then for any reachable set A in G, any valid fixing sequence on V \A gives the same CADMG over A, and the same kernel q A (x A |x V \A ) obtained from p(x V ).

A Möbius Parameterization of Binary Nested Markov Models
We now give the original parameterization for binary nested Markov models. The approach generalizes in a straightforward way to finite discrete state spaces. Multivariate binary distributions in the nested Markov model for an ADMG G may be parameterized by the following:

Definition 9
The nested Möbius parameters associated with a CADMG G are a set of functions: Q G ≡ q S (X H = 0 | x tail(H) ) for H = rh(S), S ∈ I(G) .
Intuitively, a parameter q S (X H = 0|x tail(H) ) is the probability that the variable set X H assumes values 0 in a kernel obtained from p(x V ) by fixing X V \S , and conditioning on X tail(H) . As a shorthand, we will denote the parameter q S (X H = 0|x tail(H) ) by θ H (x tail(H) ).
where the empty product is defined to be 1.
For example, the graph shown in Fig. 3 (b) representing a model over binary random variables X 1 , X 2 , X 3 , X 4 is parameterized by the following sets of parameters: The total number of parameters is 1+2+2+2+4+2 = 13, which is 2 fewer than a saturated parameterization of a 4 node binary model, which contains 2 4 − 1 = 15 parameters. The two missing parameters reflect the fact that θ 4 (x 3 ) does not depend on x 1 , which is a constraint induced by the absence of the edge from 1 to 4 in Fig. 3 (b). Note that this constraint is not a conditional independence. In fact, no conditional independences over variables corresponding to vertices 1, 2, 3, 4 are advertised in Fig. 3 (b).
This parameterization maps θ H parameters to probabilities in a CADMG via the inverse Möbius transform given by (3), and generalizes both the standard Markov parameterization of DAGs in terms of parameters of the form p(x i = 0 | pa(x i )), and the parameterization of bidirected graph models given in [3].

A Log-linear Parameterization of Nested Markov Models
We begin by defining a set of objects which are functions of the observed density, and which will serve as our parameters.
Definition 11 Let G(V, E) be an ADMG and p(x V ) a density over a set of binary random variables X V in the nested Markov model of G. For any S ∈ I(G), to be the nested log-linear parameter associated with A in S. Further let Λ(G) be the collection This parameterization is based on the graphical concepts of recursive heads and corresponding tails. We call the parameterization 'nested ingenuous' due to its similarity to a marginal log-linear parameterization called ingenuous in [6], and in contrast to other loglinear parameterizations which may exist for nested Markov models. Marginal model parameterizations of this type were first introduced in [2]. This definition extends easily to non-binary discrete data, in which case some parameters λ M A become collections of parameters.
As an example, consider the graph shown in Fig. 3 (b) which represents a binary nested Markov model. The nested ingenuous parameters associated with the marginal p(x 1 ) and conditional p(x 2 |x 1 ) are whereas parameters associated with the kernel q 4 (x 4 |x 3 ) = x2 p(x 4 |x 3 , x 2 , x 1 )p(x 2 |x 1 ) are A parameter λ M A , where M is the union of a head H and its tail T , can be viewed, by analogy with similar clique parameters in undirected log-linear models, as a |A|-way interaction between the vertices in A, within the kernel corresponding to M . For instance the kernel q 2,4 (x 2 , x 4 |x 1 , x 3 ) = p(x 4 |x 3 , x 2 , x 1 ) p(x 2 |x 1 ), 2 makes an appearance in 4 parameters in a binary model: λ 1234 24 , λ 1234 124 , λ 1234 234 , and λ 1234 1234 . If we set λ 1234 1234 to 0, we claim there is no 4-way interaction between X 1 , X 2 , X 3 , X 4 in the kernel.
It can be shown that while the Möbius parameterization of the graph in Fig. 3 (b) is variation dependent, the nested ingenuous parameterization of the same graph is variation independent. This is not true in general. In particular both parameterizations for the graph in Fig. 1 (b) are variation dependent.

Main Results
In this section we prove that the nested ingenuous parameters indeed parameterize discrete nested Markov models. We start with an intermediate result.

The Main Result
We now define a partial order on heads and use this order to inductively establish the main result.
Definition 13 Let ≺ I(G) be the partial order on heads, H i , of intrinsic sets, S i , in G such that H i ≺ I(G) H j whenever S i ⊂ S j .

Theorem 14
The nested ingenuous parameterization of an ADMG G with nodes V parameterizes precisely those distributions p(x V ) obeying the nested global Markov property with respect to G.
Proof: Let ≺ I(G) be the partial ordering on heads given in Definition 13. We proceed by induction on this ordering. For the base case, we know that singleton heads {h} with empty tails are parameterized by λ h h . If a singleton head has a non-empty tail, the conclusion follows immediately by Lemma 12. Now, suppose that we wish to find the kernel with a non-singleton head H † and a tail T † corresponding to the intrinsic set S † . Assume, by inductive hypothesis, that we have already obtained the kernels with all heads H such that H ≺ I(G) H † . We claim this is sufficient to give the ( Fix a particular v ∈ H † . The set S † \ {v} is reachable, since V \ S † is a set with a valid fixing sequence, and any v ∈ H † has no children in S † in φ V \S † (G) so is fixable in φ V \S † (G). Theorem 7 and Theorem 8 imply that for every reachable set A, (2) holds. Hence: For any S such that rh G (S) = H, and H ⊆ S † \ {v}, H ≺ I(G) H † , hence by the induction hypothesis, the kernel q S (x S |x pa(S)\S ) is already obtained, and all kernels which appear in (4) can be derived by conditioning from some such q S (x S |x pa(S)\S ). The desired kernel q S † (x H † \{v} |x T † ) can itself be obtained from q S † (x S † \{v} |x pa(S † )\S † ) by conditioning.
We can repeat this argument for any v ∈ H † . Finally, the nested ingenuous parameterization contains The result then follows by Lemma 12.

Simulations
To illustrate the utility of setting higher order parameters to zero ('removing'), we present a simulation study based on the ADMG in Fig. 5 (b). This graph is a special case of two bidirected chains of k vertices each, with a path of directed edges alternating between the chains, for k = 4. The number of parameters in the relevant binary nested Markov model grows exponentially with k in graphs of this type.
Consider also the latent variable model defined by replacing each bidirected edge with an independent latent variable shown in Fig. 5 (a), so that 1 ↔ 3 becomes 1 ← 9 → 3. If the state space of each latent variable is the same and fixed, then the number of parameters in this hidden variable DAG model grows only linearly in k. This suggests that the nested Markov model may include higher order parameters which are not really necessary in this case (though the higher order parameters may become necessary again if the state space of latent variables grows).
We generated distributions from the latent variable model associated with the DAG in Fig. 5 (a) as follows: each of the six latent variables takes one of three states with equal probability, and each observed variable takes the value 0 with a probability generated as an independent uniform random variable on (0, 1), conditional upon each possible value of its parents.
For each of 1,000 distributions produced independently using this method, we generated a dataset of size 5,000. We then fitted the nested model generated by the graph in Fig. 5 (b) to each dataset by maximum likelihood estimation, using a variation of an algorithm found in [5], and measured the increase in deviance associated with zeroing any nested ingenuous parameters corresponding to effects above a certain order. If these parameters were truly zero, we would expect the increase to follow a χ 2 -distribution with an appropriate number of degrees of freedom; the first two histograms in Fig. 4 demonstrate that the distribution of actual increases in deviance looks much like the relevant χ 2 -distribution if we remove interactions of order 6 and higher. The third histogram shows that this starts to break down slightly when 5-way interactions are also zeroed.
These results suggest that higher order parameters are often not useful for explaining finite datasets, and more parsimonious models can be obtained by removing them; a similar simulation was performed for the Markov case in [6].

Distinguishing Graphs
The use of score-based search methods for recovering nested Markov models had been investigated [15]. It was found that relatively large sample sizes were required to reliably recover the correct graph, even in examples with only 4 or 5 binary nodes and after ensuring that the underlying distributions were approximately faithful to the true graph. One phenomenon identified was that incorrect but more parsimonious graphs, especially DAGs, tended to have lower BIC scores than the correct models, which include higher order parameters. Although BIC is guaranteed to be smaller on the correct model asymptotically, in finite samples it applies strong penalties for having additional parameters with little explanatory power.
Here we present a simulation to show how the new parameterization can help to overcome this difficulty. Using the method described in the previous subsection, we generated 1,000 multivariate binary distributions which were nested Markov with respect to the graph in Fig. 1 (b). For each distribution we generated a dataset, and fitted the data to the correct model, which has 16 parameters, as well as the two DAGs given in Fig. 7 (a) and (b), which each have  Figure 6: From the experiment in Section 7.1: in red, the proportion of times graph in Fig. 1 (b) had lower BIC than the DAGs in Fig. 7, for varying sample sizes; in black, the proportion of times some restricted version of this model had a lower BIC than any restricted versions of either DAG. 11 parameters. This was repeated at various sample sizes.
The plot in Fig. 6 shows, in red, the proportion of times in which the BIC score for the correct model was lower than that for each of the DAGs, at various sample sizes. The correct graph only has the lowest BIC score of the three graphs on less than 3% of runs at sample size of n = 1,000, increasing to around 50% for n = 20,000.
In addition to the full models, we fitted the datasets to versions of the models with higher order parameters removed; the graph in Fig. 1 (b) can be restricted by zeroing the 5-way parameter (leaving 15 free parameters), the 4-way and and above (13 params), or 3-way and above (10 params). Similarly we can restrict the DAGs to have no 3-way effects, giving each model 10 free parameters. Fig. 6 shows, in black, the proportion of times that one of these restricted versions of the true model had a lower BIC than any version of either DAG model. We see that the correct graph has the lowest score in 40% of runs at n = 1,000, rising to around 70% at n = 20,000. Note that these results should not be compared directly to those in [15], since the single ground truth law used in that paper was generated so as to ensure faithfulness to the correct graph, whereas we are randomly sampling multiple laws without bothering to ensure any particular properties in these laws other than consistency with the underlying DAG.
These results suggest that these submodels of the nested model may be advantageous in recovering the correct graphical structure using score-based methods.  Fig. 1 (b).
Note that determining which higher order parameters should be set to zero for a given data set and sample size remains non-trivial. Automatic selection might be possible with an L 1 -penalized approach [16,4].

Discussion and Conclusions
We have introduced a new log-linear parameterization of nested Markov models over discrete state spaces. The log-linear parameters correspond to 'interactions' in kernels obtained after an iterative application of truncation and marginalization steps (informally 'interactions in interventional densities'). By contrast the Möbius parameters [15] correspond to context specific effects in kernels (informally 'context specific causal effects').
We have shown by means of a simulation study that in cases where data is generated from a marginal of a DAG with 'weak confounders', we can reduce the dimension of the model by ignoring higher order interaction parameters, while retaining the advantages of nested Markov models compared to modeling weak confounding directly in a DAG.
Though there is no efficient, closed form mapping from ingenuous parameters to either Möbius parameters or standard probabilities, this is a smaller disadvantage than it may seem. This is because in cases where the ingenuous parameterization was used to select a particular submodel based on a dataset, we may still reparameterize and use Möbius parameters, or even standard joint probabilities if desired. Moreover, this reparameterization step need only be performed once, compared to multiple calls to a fitting procedure which identified the particular graph corresponding to our submodel in the first place.
The ingenuous and the Möbius parameterizations are thus complementary. The natural application of the ingenuous parameterization is in learning graph structure from data in situations where many samples are not available, but we expect most confounding to be weak. The natural application of the Möbius parameterization is the support of probabilistic and causal inference in a particular graph [14,15], in cases where an efficient mapping from parameters to joint probabilities is important.