{'fraction_non_alphanumeric': 0.02811252875144986, 'fraction_numerical': 0.010733874614189946, 'mean_word_length': 4.809502055733211, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modelling contextuality by probabilistic programs with hypergraph semantics

Models of a phenomenon are often developed by examining it under different experimental conditions, or measurement contexts. The resultant probabilistic models assume that the underlying random variables, which define a measurable set of outcomes, can be defined independent of the measurement context. The phenomenon is deemed contextual when this assumption fails. Contextuality is an important issue in quantum physics. However, there has been growing speculation that it manifests outside the quantum realm with human cognition being a particularly prominent area of investigation. This article contributes the foundations of a probabilistic programming language that allows convenient exploration of contextuality in wide range of applications relevant to cognitive science and artificial intelligence. Specific syntax is proposed to allow the specification of"measurement contexts". Each such context delivers a partial model of the phenomenon based on the associated experimental condition described by the measurement context. The probabilistic program is translated into a hypergraph in a modular way. Recent theoretical results from the field of quantum physics show that contextuality can be equated with the possibility of constructing a probabilistic model on the resulting hypergraph. The use of hypergraphs opens the door for a theoretically succinct and efficient computational semantics sensitive to modelling both contextual and non-contextual phenomena. Finally, this article raises awareness of contextuality beyond quantum physics and to contribute formal methods to detect its presence by means of hypergraph semantics.


Introduction
Probabilistic models are used in a broad swathe of disciplines ranging from the social and behavioural sciences, biology, the physical and computational sciences, to name but a few. At their very core, probabilistic models are defined in terms of random variables, which range over a set of outcomes that are subject to chance. For example, a random variable R might be defined to model the performance of human memory. In this case, the possible outcomes might be words studied by a human subject before their memory is cued. After cueing, the subject recalls the first word that comes to mind from the set of study words. This outcome is recorded as a measurement. Repeated measurements over a set of subjects allow the probability of the recall of a certain word to be empirically established.
It is important to note from the outset that the random variable R has been been devised by the modeller with a specific functional identity in mind, namely to model the recall of a set of predefined study words. When developing probabilistic models in this way, the underlying assumption is that the functional identity of a random variable is independent of the context in which it is measured. For example, the purpose, or functional identity of R is assumed to be the same regardless of whether the memories of human subjects are studied in a controlled laboratory, or in "the wild", such as in a night club. This assumption seems perfectly reasonable. However, in quantum physics the analog of this assumption does not always hold and has become known as "contextuality". More formally, the Kochen-Specker theorem (Kochen and Specker, 1967) implies that quantum theory is incompatible with the assumption that measurement outcomes are determined by physical properties that are independent of the measurement context. Placing this theorem in the context of probabilistic models: contextuality is the "impossibility of assigning a single random variable to represent the outcomes of the same measurement procedure in different measurement conditions" (Acacio De Barros and Oas, 2015).
Contextuality plays a central role in the rapidly developing field of quantum information in delineating how quantum resources can transcend the bounds of classical information processing (Howard et al., 2014). It also has important consequences for our understanding of the very nature of physical reality. It is still an open question, however, if contextuality manifests outside the quantum realm. Some authors in the emerging field of quantum cognition have investigated whether contextuality manifests in cognitive information processing, for example, human conceptual processing (Gabora and Aerts, 2002;Bruza et al., 2015;Gronchi and Strambini, 2016) and perception (Atmanspacher and Filk, 2010;Asano et al., 2014;Zhang and Dzhafarov, 2017).
It is curious that the preceding deliberations around random variables have a parallel in the field of computer programming languages. More than five decades ago, programming languages such as FORTRAN featured variables that were global. (In early versions of FORTAN, all variables were global.) As programming languages developed, global variables were seen as a potential cause of errors. For example, in a large program a variable X can inadvertently be used for functionally different purposes at different points in the program. The error can be fixed by splitting variable X into two global variables X 1 and X 2 . In this way X 1 can be used for one functional purpose and X 2 for the other, and hence there is no danger that their unique functional identities can become confounded. However, when the program involves large numbers of global variables, keeping track of the functional identities of variables can became tedious and a source of error. Such errors were considered serious and prevalent enough that following in the wake of Dijkstra's famous paper titled "Go To statement considered harmful", Wulf and Shaw (1973) advocated in a similarly influential article that global variables are "harmful" and perhaps should be abolished. This stance was developed in relation to block structured programming languages. A "block", or "scope", refers to the set of program constructs, such as variable definitions, that are only valid within a delineated syntactic fragment of program code. Wulf and Shaw (1973) argued that when a program employs a scope in which variable X is defined locally, as well as a variable with the same label X that is global to that scope, then X becomes "vulnerable" for erroneous overloading. The theory of programming languages subsequently developed means so that a variable with the same label can be used in two different scopes but preserve a unique functional identity within the given scope. This is not the case in state-of-the-art probabilistic modelling. We believe that the way probabilistic models are currently developed is somewhat akin to writing FORTRAN programs from a few decades ago. By this we mean that in the development of a probabilistic model all the random variables are global. As a consequence errors can appear in the associated model should the functional identity of variables be changing because the phenomenon being modelled is contextual.
The aim of this article to contribute the foundations of a probabilistic programming language that allows convenient exploration of contextuality in wide range of applications relevant to cognitive science and artificial intelligence. For example, dedicated syntax is illustrated which shows how a measurement context can be specified as a syntactic scope in a probabilistic program. In addition, random variables can be declared local to a scope to allow overloading, which is convenient for the development of models. Such programs are referred to a P-programs and fall within the emerging area of probabilistic programming (Gordon et al., 2014).
Probabilistic programming languages (PPLs) unify techniques from conventional programming such as modularity, imperative or functional specification, as well as the representation and use of uncertain knowledge. A variety of PPLs have been proposed (see Gordon et al. (2014) for references), which have attracted interest from artificial intelligence, programming languages, cognitive science, and the natural language processing communities (Goodman and Stuhlmüller, 2014). However, unlike conventional programming languages, which are written with the intention to be executed, a core purpose of a probabilistic program is to specify a model in the form of a probability distribution. In short, PPLs are high-level and universal languages for expressing probabilistic models. As a consequence, these languages should not be confused with probabilistic algorithms, or randomized algorithms, which employ a degree of randomness as part of their logic.
In addition to the dedicated syntax, P-programs have a semantics based on hypergraphs which determine whether the phenomenon is contextual. These semantics will be based on hypergraphs in two ways: Firstly, a hypergraph approach from relational database theory is used to determine whether the schema of variables of the various measurement contexts is acyclic. If so, the phenomenon is being modelled is non-contextual. Secondly, if the schema is cyclic, measurement contexts are mapped to "contextuality scenarios", which are probabilistic hypergraphs. Although these hypergraphs have been developed in the field of quantum physics, they provide a comprehensive general framework to determine whether the phenomenon being modelled by the P-program is contextual.

An example P-program
In order to convey some of the core ideas behind P-programs, Figure 1 illustrates an example program where the phenomenon being modelled is two coins being tossed in four experimental conditions. Some of these conditions induce various biases on the coins. The syntax of the P-program is expressed in the style of a feature rich probabilistic programming language called WebPPL 1 (Goodman and Tenenbaum, 2016). However, the choice of the language is not significant. WebPPL is simply being used as an example syntactic framework.
In P-programs, syntactic scopes are delineated by the reserved word context. Each scope specifies an experimental condition, or "measurement context" under which a phenomenon is being examined. The example P-program defines four such contexts labelled P1, P2, P3 and P4. Consider context P1 which declares two coins as dichotomous random variables A1 and B1 which are local to this scope. The syntax flip(0.5) denotes a fair coin; any value other than

P1
: Figure 2: Four p-tables returned by the respective contexts P1, P2, P3, P4 from the P-program of Figure 1. The values in the column labelled "p" denote probabilities and sum to unity in each table.
0.5 defines a biased coin. Declaring variables local to the scope syntactically expresses the assumption that the variables retain a unique functional identity within the scope. The random variable declarations within a scope define a set of events which correspond to outcomes which can be observed in relation to the phenomenon being examined in the given measurement context. For example, A1 = 1 signifies that coin A1 has been observed as being a head after flipping. Joint event spaces are defined by the syntax p[A1,B1] which becomes a joint probability distribution by the syntax Infer(samples:1000,p). In this case two coins have been flipped 1000 times to prime the probabilities associated with each of the four mutually exclusive joint events in the event space. The resulting distribution represents the model of the phenomenon in that measurement context which is returned from the scope as a partial model. The other measurement contexts P2, P3 and P4 are similarly defined resulting in the four distributions depicted in Figure 2. The structure of each distribution will be referred to as a probabilistic table, or p-table for short, as these are a natural probabilistic extension to the tables defined in relational databases (Bruza and Abramsky, 2016). Modelling practice is usually governed by the norm that it is desirable to construct a single model of the phenomenon being studied. Dedicated syntax, e.g., model(P1,P2,P3,P4) allows partial models from the four measurement contexts to be combined to form a single distribution, such that each distribution corresponding to partial models can be recovered from this single distribution by appropriately marginalizing it (Bruza and Abramsky, 2016;Bruza, 2016). It turns out that this is not always possible to construct such a single model. As we shall see below, when this happens the phenomenon being modelled turns out to be "contextual". Abramsky (2015) discovered that this formulation of contextuality is equivalent to the universal relation problem in database theory. This problem involves determining whether the relations in a relational database be joined together (using the natural join operator) to form a single "universal" relation such that each constituent relation be can be recovered from the universal relation by means of projection.
Relational database theory tells us that a key consideration in this problem turns out to be whether the database schema comprising constituent relations (p-tables) is cyclic or acyclic. A database schema is deemed "acyclic" iff the hypergraph H(N, E) can be reduced into an empty graph using the Graham procedure (Gyssens and Paredaens, 1984). The set N denotes the vertices in the graph and E the set of edges.
A hypergraph differs from a normal graph in that an edge can connect more than two vertices. For this reason, such edges are termed "hyperedges". For example, the database schema corresponding to the P-program in Figure 1 is depicted in Figure 2. In our case, the nodes N of the hypergraph are the the individual variables in the headers of the p-tables and the edges correspond to the sets of variables in these headers, i.e., there will be one edge corresponding to each constituent p-table, where the edge is the set of variables defining the header of that p-table. Therefore, (As the headers of p-tables only contain two variables, H is in this case a standard graph.) The Graham procedure is applied to the hypergraph H until no further action is possible: • delete every edge that is properly contained in another one; • delete every node that is only contained in one edge.
The following details the steps of the Graham procedure when applied to the example: In this case the Graham procedure results in an empty hypergraph, so the schema is deemed "acyclic".
There are a number of theoretical results in relational database theory which make acyclic hypergraphs significant with regard to providing the semantics of joining partial models into a single model. Wong (1997) formalizes the relationship between Markov distributions and relational database theory by means of a generalized acyclic join dependency (GAJD). The key idea behind this relationship is the equivalence between probabilistic conditional independence and a generalized form of multivalued dependency, the latter being a functional constraint imposed between two sets of attributes in the database schema. It turns out that a joint distribution factorized on an acyclic hypergraph is equivalent to a GAJD (Wong, 2001).
For example, consider once again the acyclic schema in Figure 2. There are four p-tables, P = {P 1 , P 2 , P 3 , P 4 }. As the hypergraph is acyclic, there is necessarily a so called join tree construction, denoted ⊗{S 1 , . . . , S n }, that satisfies the GAJD. In the tree construction, each S i , 1 ≤ i ≤ n denotes a unique p-table in the set P . The practical consequence of this is that there is a join expression of the form: (((S 1 ⊗ S 2 ) ⊗ S 3 ) ⊗ S 4 ) where the sequence S 1 , . . . , S 4 is a tree construction ordering derived from the acyclic hypergraph. If the hypergraph constructed from the schema comprising n p-tables {P 1 , P 2 , P 3 , . . . P n } is acyclic, then a generalized join expression (. . . (S 1 ⊗ S 2 ) ⊗ S 3 ) . . . ⊗ S n ) exists which joins the p-tables into a single probability distribution P such that each Figure 3: Join tree of the p-tables P 1 , P 2 , P 3 , P 4 to be joined.
In order to gain some intuition about how this plays out in practice, the acyclic database schema depicted in Figure 2 results in the join tree depicted in Figure 3. The nodes depict the variables in the respective p-tables and the edges represent the overlap between the sets of variables in the respective headers. The numbers on the edges denote the ordering used to produce the join expression: (((P 3 ⊗P 1 )⊗P 2 )⊗P 4 ). Under the assumption that the probability distributions represented in the nodes have identical distributions when marginalized by the variable associated with the edge, we can see how the hypertree produces a Markov network which, in turn, specifies the probabilistic join of the constituent p-tables (Liu et al., 2011): Observe how the structure of the equation mirrors the graph in Figure 3 where the numerator corresponds to the nodes of the join tree and the denominator corresponds to terms which normalize the probabilities. In addition, this expression reflects conditional independence assumptions implied by the join tree, namely A 1 and A 2 are conditionally independent of B 1 , B 1 and B 2 are conditionally independent of A 1 , and A 1 and A 3 are conditionally independent of B 2 .
Let us summarize the situation so far and reflect on the issue of contextuality. A P-program comprises a number of scopes where each scope corresponds to a measurement context. A scope returns a probability distribution in the form of a p-table, which can be considered a partial model of the phenomenon. A reasonable goal is to combine these distributions into a single distribution so a single model of the phenomenon is produced. When the schema of the constituent p-tables is acyclic and the marginal distributions of the set of intersecting variables are constant, then a straightforward extension of relational database theory can be used to produce the required single model (as has been shown in more detail in (Bruza, 2016)). The fact that it is possible to construct a single model means that the random variables in the P-program have a functional identity that is independent of the measurement contexts. The phenomenon is therefore non-contextual.
Much of the research on contextuality corresponds to when the schema of the p-tables is cyclic (Zhang and Dzhafarov, 2017). In order to explore such cases, we will continue to use hypergraphs, but instead on defining the graph structure at the level of the schema as was illustrated in Graham procedure, the structure of the hypergraph will be defined in terms of the underlying events in the measurement contexts defined in the P-program. In database terms this equates to defining the hypergraph structure in terms of the data in the respective p-tables.

Probabilistic models and Hypergraph semantics
In the following, we draw from a comprehensive theoretical investigation using hypergraphs to model contextuality in quantum physics (Acin et al., 2015). The driving motivation is to leverage these theoretical results to provide the semantics of P-programs when the schema of the p-tables to be joined is cyclic. How these semantics are expressed relates to how the syntax has been specified, which in turn relates to the experimental design that the modeller has in mind. The basic building block of these semantics is a "contextuality scenario".
Definition 3.1. (Contextuality Scenario) (Definition 2.2.1 (Acin et al., 2015)) A contextuality scenario is a hypergraph X = (V, E) such that: • v ∈ V denotes an event which can occur in a measurement context; • e ∈ E is the set of all possible events in a measurement context.
The set of hyperedges E are determined by both the measurement contexts as well as the measurement protocol. Each measurement context is represented by an edge in the hypergraph X. The basic idea is that each syntactic scope in a P-program will lead to a hyperedge, where the events are a complete set of outcomes in the given measurement context specified in the associated scope. Additional hyperedges are a consequence of the constraints inherent in the measurement protocol that is applied. The examples to follow aim to make this clear. In some cases, hyperedges will have a non-trivial intersection: If v ∈ e 1 and v ∈ e 2 , then this represents the idea that the two different measurement outcomes corresponding to v should be thought of as equivalent as will be detailed below by means of an order effects experiment.
Order effects experiments involve two measurement contexts each involving two dichotomous variables A and B which represent answers to yes/no questions Q A and Q B . In one measurement context, the question Q A is served before question Q B and in the second measurement context the reverse order is served, namely Q B then Q A . Order effects occur when the answer to the first question influences the answer to the second. These two measurements contexts are syntactically specified by the scopes P1 and P2 shown in Figure 4.
In this P-program, syntax of the form var B = A ? flip(0.8): flip(0.1) models the influence of the answer of Q A on Q B via a pair of biased coins. In this case, if Q A = y, then the response to Q B is determined by flipping an 80% biased coin. Conversely, if Q A = n, then the response to Q B is determined by flipping a 10% biased coin (The choices of such biases are determined by the modeller). It should be carefully noted that the measurement contexts in the order effects program do not reflect the usual understanding of measurement context employed in experiments analyzing contextuality in quantum physics.
In these experiments, a measurement context comprises observables that are jointly measurable, so the order in which the observables within a given context are measured will not affect the associated statistics.
We will now use this simple example to illustrate the associated contextuality scenario which is shown in Figure 5. Firstly, the set of V of events (measurement outcomes) comprises all possible combinations of yes/no answers to the questions Q A and Q B , namely , where 1 denotes 'yes' and 0 denotes 'no'. In this figure, the two rounded rectangles represent the events within the two measurement contexts specified by the syntactic scopes P1 and P2. For example, in the rectangle labeled P1, "11" is shorthand for the event A = 1 ∧ B = 1 etc. Observe that the corresponding hyperedges (rounded rectangles) contain an exhaustive, mutually exclusive set of events. In addition, the two spanning hyperedges going across these rectangles similarly comprise a set of exhaustive, mutually exclusive set of events. These spanning edges help illustrate events that are considered to be equivalent. Firstly, it is reasonable to assume answering yes (or no) to both questions in either measurement context represents equivalent events. Therefore, the events labelled p 1 and p 4 can respectively be assumed equivalent to q 1 and q 4 . It becomes a little more subtle when the polarity of the answers differ. For example, the event labelled p 3 represents the event A = 0∧B = 1, remembering that question Q A was asked before question Q B in this context. The equivalent event in hyperedge P2 is labelled q 2 , which corresponds the event B = 1 ∧ A = 0, where question B is asked before question A. As conjunction is commutative, it is reasonable to view these two converse events as equivalent. In summary, if p 3 is equivalent to q 2 and p 4 is equivalent to q 4 then the hyperedge {p 1 , p 2 , q 2 , q 4 } (the dashed hyperedge in Figure 5) can be established, in addition to the hyperedge {p 1 , p 2 , p 3 , p 4 }.
Let us now return to the issue of contextuality. A probabilistic model corresponding to a contextuality scenario X is the mapping of measurement outcomes to a probability p : V → [0, 1]. Henson and Sainz (2015) point out that "By defining probabilistic models in this way [rather than by a function p e (V ) depending on the measurement e performed], we are assuming that in the set of experimental protocols that we are interested in, the probability for a given outcome is independent of the measurement that is performed".
Observe carefully that by defining probabilistic models in this way formalizes the assumption mentioned in the introduction, namely that random variables are independent of measurement context and thus have a single functional identity. Without a single functional identity it is impossible to assign a random variable to represent the outcomes of the same measurement protocol in different measurement contexts.
It is a requirement that the mapping adheres to the expected normalization condition: ∀ e∈E : v∈e p(v) = 1. By way of illustration, consider once again Figure 5. This contextuality scenario has four edges. The normalization condition enforces the following constraints: p 1 + p 2 + q 2 + q 4 = 1 (4) where p i , 1 ≤ i ≤ 4 and q j , 1 ≤ j ≤ 4 denote the probabilities of outcomes in the four hyperedges. A definition of contextuality can now be presented.
Definition 3.2 ((Probabilistic) contextuality). Let X = (V, E) be a contextuality scenario. Let G(X) denote the set of probabilistic models on X. X is deemed "contextual" if G(X) = ∅.
In other words, the impossibility of a probabilistic model signifies that the phenomenon being modelled is contextual. (The label "probabilistic" mirrors an analogous definition of contextuality based on sheaf theory ).
Let us now examine the possibility of a probabilistic model on the order effects contextuality scenario ( Figure 5). Equations (2) and (5) imply that p 1 + p 2 = q 1 + q 3 . Now, p 1 + p 2 are repectively associated with the outcomes A = 1 ∧ B = 1 and A = 1 ∧ B = 0. In other words, p 1 + p 2 denotes the marginal probability p(A) in measurement context P1. By a similar argument, q 1 + q 3 denotes p(B = 1 ∧ A = 1) + p(B = 0 ∧ A = 1) which is written this way to emphasize that question Q B is asked first in measurement context P2. This also equates to the marginal probability p(A). In other words, the constraints imposed by normalization conditions in the hyperedges imply that the marginal probability p(A) must be the same across both measurement contexts P1 and P2.
This conclusion makes sense when considered in relation to the definition of contextuality: The only way that a function p : V → [0, 1] can be defined is if the marginal probabilities of the variables A and B are the same in both measurement contexts P1 and P2. If not, then this means that variable A has a different functional identity when question Q A is asked first (in measurement context P1) as opposed to when it is asked second (in measurement context P2).
In summary, the semantics of a P-program is represented by a contextuality scenario, which has the form of a hypergraph. Contextuality equates to the impossibility of a probabilistic model over the hypergraph. This impossibility is where contextuality meets probabilistic models.

Syntax and semantics of combining contextual scenarios according to experimental design
Different fields employ various experimental designs when studying a phenomenon of interest. For example, in psychology a "between subjects" ex-perimental design means a given participant should only be subjected to one measurement context. In quantum physics, however, some experiments involve measurement contexts which are enacted simultaneously with the requirement that observations made in each context are local to that context and don't influence other measurement contexts. This constraint is often referred to as the "no signalling" condition.
One of the advantages of using a programming approach to develop probabilistic models is that experimental designs can be syntactically specified in a modular way. In this way, a wide variety of experimental designs across fields can potentially be catered for. For example, consider the situation where an experimenter wishes to determine whether a system S can validly be modelled compositionally in terms of two component subsystems A and B as shown in  Two different experiments can be carried out upon each of the two presumed components, which will answer a set of 'questions' with binary outcomes, leading to four measurement contexts. For example, one experimental context would be to ask A1 of component A and B1 of component B.
This abstract experimental design has be instantiated in a number of ways. For example, in quantum physics it has been employed to determine whether system S comprising photons 2 . A and B is entangled. In addition, it has been employed in cognitive psychology to test for contextuality in human cognition Bruza et al., 2015;Gronchi and Strambini, 2016). For example, Bruza et al. (2015) describe an experiment to determine whether novel conceptual combinations such as BOXER BAT adhere to the principle of semantic compositionality (Pelletier, 1994). Semantic compositionality entails that the meaning of BOXER BAT is some function of the meaning of the component concepts BOXER and BAT. In this case, component A corresponds to the concept BOXER and component B corresponds to the concept BAT. Each of these concepts happen to be biambiguous, for example, BOXER can be interpreted in a sport sense or an animal sense (a breed of dog). Similarly, the concept BAT can be interpreted in either of these senses. Interpretation of concepts can be manipulated by priming words which correspond to the 'questions' asked of the component concepts. For example, one experimental context would be to ask a set of human subjects to return an interpretation of BOXER BAT after being shown the priming words fighter (A1) and vampire (B1). Note how A1 is designed to prime the sport sense of BOXER and B2 to prime the animal sense of BAT. An interpretation given in this context might be "an angry furry black animal with boxing gloves on". It is important to note that the interpretation is probabilistic, namely the priming word influences an interpretation of the concept but does not determine it.
How can system S depicted in Figure 6 be modelled as a P-program? And, how can the semantics of the P-program determine whether S is contextual? One way to think about system S is that it is equivalent to a set of biased coins A and B, where the bias is local to a given measurement context. Figure 7 depicts a P-program that follows this line of thinking.
The P-program will be referred to as "Bell scenario" as it programmatically specifies the design of experiments in quantum physics inspired by the physicist John Bell (Clauser and Horne, 1974). Such experiments involve a system of two space-like separated photons.

Bell contextuality scenario with no-signalling
The Bell scenario program follows the design depicted in Figure 6 by first defining the components A and B together with the associated variables. Thereafter, the program features the four measurement associated contexts P1, P2, P3 and P4. Finally, the line model(design: 'no-signal',P1,P2,P3,P4) specifies that the measurement contexts are to be combined according to a "no signalling" condition. Formal details of this condition will follow, but essentially it imposes a constraint that measurements made on one component do not affect outcomes observed in relation to the other component. This could be because the components have sufficient spatial separation in a physics experiment, or alternatively, in a psychology experiment the cognitive phenomena represented by components A and B of system S are independent cognitive functions.
The question now to be addressed is how the hypergraph semantics are to be formulated. Acin et al. (2015) provides the general semantics of the Bell scenarios by means of multipartite composition of contextuality scenarios. As these semantics are compositional, it opens the door to map syntactically specified components in a P-program to contextuality scenarios and then to exploit the composition to provide the semantics of the program as a whole. Consider the Bell scenario program depicted in Figure 7. The syntactically defined components A and B are modelled as a contextuality scenarios X A and X B respectively. The corresponding hypergraphs are depicted in Figure 8. Note how the variable definitions associated with the component map to an edge in a hypergraphs. For example, the syntax def A = component(A1,A2) corresponds to the two edges labelled A1 and A2 on the left hand side of Figure  8.
The question now is how the compose the contextuality scenarios X A and X B into a single contextuality scenario X AB , which will express the semantics of the Bell scenario P-program. The most basic form of composition is by means of a direct product of the respective hypergraphs. The direct product is a contextual scenario (Acin et al., 2015).) The hypergraph of the product is shown in Figure 10. Observe how each syntactic context P1, P2, P3 and P4 specified in the Bell scenario P-program corresponds to an edge in the hypergraph. In addition, note the structural correspondence of the hypergraph in Figure 10 with the cyclic database schema depicted in Figure 9. Note that the events in Figure 10 are denoted as various coloured dots with each such dot corresponding directly to a row of a p-table within the cyclic schema.
The Bell scenario program syntactically specifies that there should be "no signalling" between the respective components A and B via the command model(design: 'no-signal',P1,P2,P3,P4). This condition imposes constraints on the allowable probabilistic models on the combined hypergraph struc-  (Acin et al., 2015), a probabilistic model p ∈ G(X A × X B ) is a "no signalling" model if: The probabilistic constraints entailed by this definition will be illustrated in an example to follow. Acin et al. (2015)(p45) show that not all probabilistic models of contextuality scenarios composed by a direct product are "no signalling" models. In order to guarantee that all probabilistic models of a combined contextuality scenario are "no signalling" models, the constituent contextuality scenarios X A and X B should be combined by the Foulis-Randall (FR) product denoted X AB = X A ⊗ FR X B . As with the direct product X A × X B of contextuality scenarios, the vertices of the FR product are defined by . It is with respect to the hyperedges that there is a difference between the FR product and the direct product: We are now in a position to illustrate the semantics of the P-program of Figure  7 by the corresponding contextuality scenario depicted in Figure 11. Observe how the FR product produces the extra edges that span the events across measurement contexts labeled P1, P2, P3 and P4 when compared with the direct product hypergraph depicted in Figure 10, At first these spanning edges may seem arbitrary, but they happen to guarantee that the allowable probabilistic models over the composite contextuality scenario X A ⊗ FR X B satisfy the "no signalling" condition (Sainz and Wolfe, 2017). By way of illustration, the normalization condition on edges imposes the following constraints (see Figure  11): p 1 + p 2 + q 3 + q 4 = 1 (8) where p i , 1 ≤ i ≤ 4 and q j , 1 ≤ j ≤ 4 denote the probabilities of events in the respective hyperedges. A consequence of constraints (6) and (8) is that p 3 + p 4 = q 3 + q 4 . When considering the associated outcomes this means (The preceding is an example of one of the constraints imposed by Definition 3.1.2 in (Acin et al., 2015) as specified above). In other words, the marginal probability p(A1 = 1) does not differ across the measurement contexts P1 and P2 specified in the P-program of Figure 7. In a similar vein, equations (6) and (9) imply that the marginal probability p(A1 = 0) does not differ across measurement contexts P1 and P2. The stability of marginal probability ensures that "no signalling" is occurring from component B to component A (see Figure  6). In terms of our BOXER BAT example, "no signalling" implies that the probability of interpretation of the concept BOXER does change whether the priming word for BAT is ball (B1 -sport sense) or vampire (B2 -animal sense).

Bell contextuality scenario with signalling
Investigations into contextuality in quantum physics involve the "no signalling" condition. However, in cognitive science and related areas, the situation seems isn't as clear cut.  argue, for example, that  Figure 7. This P-program has the cyclic schema depicted in Figure 9. In total the hypergraph comprises 12 edges of four events. The nodes in rectangles represent events in a probability distribution returned by a given scope:P1, P2, P3, and P4. Note this figure depicts a single hypergraph. Two copies have been made to depict the spanning edges more clearly. This figure corresponds to Figure 7f in (Acin et al., 2015).
the "no signalling" condition seems always to be violated in psychological experiments. By way of illustration, consider once again the conceptual combination BOXER BAT. Recall that the "no signalling" condition entails that the probability of interpretation of the concept BOXER does change whether the priming word for BAT is ball (B1 -sport sense) or vampire (B2 -animal sense). Nor does the probability of interpretation of the concept BAT does change whether the priming word for BOXER is fighter (A1 -sport sense) or dog (A2 -animal sense). However, it is easy to imagine that signalling may be involved in forming an interpretation of BOXER BAT. For example, Wisniewski (1997) identifies a property-based interpretation of conceptual combinations whereby properties of the modifying concept BOXER apply in some way to the head concept BAT. One way to view this kind of interpretation is that a sense of BOXER is first established and then influences the interpretation of the concept BAT. In other words, the interpretation of the conceptual combination is formed by processing the combination from left to right. In relation to the general system depicted in Figure 6, the preceding situation involves an arrow proceeding from component A to B, which represents component A signalling information to component B.
We can model Wisniewski's property interpretation by extending the Bell scenario to involve signalling as specified in the P-program shown in Figure 12. The signalling from concept A to concept B in a given measurement context is modelled as the outcome of the B coin being dependent on the outcome of the A coin. Note that signalling does not occur the other way, namely, the probability of interpretation of A does not change according to outcomes measured in relation to component B. This fact allows a more refined understanding of the hypergraph semantics depicted in Figure 11 and how these semantics relate In the previous section it was established that the spanning edges in the left hand side of this figure prevents signalling from component B to A. Conversely, the spanning edges in the right hand side of the figure prevent signalling from A to B. Therefore, the hypergraph semantics of the signalling Bell scenario specified by the program in Figure 12 does not include these right hand side spanning hyperedges. The resulting hypergraph semantics are depicted as the contextuality scenario shown Figure 13 which is the semantics corresponding to the syntax model(design: 'signal(A->B)',P1,P2,P3,P4), where A->B expresses the direction of the signalling between the respective components. Definition 3.2 can then be applied to determine whether a probabilistic model exists in relation to this contextuality scenario. If not, the signalling system modelled by the P-program in Figure 12 is deemed to be "strongly contextual".

Discussion
The aim of this article is to take an algorithmic approach for the development of probabilistic models by providing a high level language that makes it convenient for the modeller to express models of a phenomenon that may be contextual. Borrowing from programming language theory, a key feature is the use of syntactic scopes which permits measurement contexts to be specified that correspond to the experimental conditions under which the phenomenon is being examined. The use of syntactic scopes has two consequences. Firstly, random variables local to a scope will be invisible to those local to other scopes. Secondly, each scope returns a probability distribution as a partial model.
The first consequence relates to scopes preventing the incorrect overloading of random variables. This article has attempted to show that the overloading of variables in probabilistic models relates to contextuality namely "contextual situations are those in which seemingly the same random variable changes its identity depending on the conditions under which it is recorded" (Dzhafarov and Kujala, 2014a).
Regarding the second consequence, Abramsky (2015) discovered that the problem of combining partial models into a single model has an equivalent expression in relational database theory where the problem is to determine whether a universal relation exists for a set of relations such that these relations can be recovered from the universal relation via projection. Contextuality occurs when it possible to construct the universal relation. The question to be addressed, then, is how to determine when it is possible, and when it is not.
This article proposes hypergraphs as an underlying semantic structure to address this question. Firstly, an approach developed in relational database theory is used to determine whether the schema of the partial models are acyclic. If so, the hypergraph is exploited to form a join tree which can compute a single model such that the partial models can be retrieved by appropriately marginalizing this model.
When the schema is cyclic, hypergraphs called "contextuality scenarios" are formed. The general picture is the following: Experimental designs are syntactically specified in addition to associated measurement contexts appropriate to the design. Each component can be translated into a contextuality scenario. Multipartite composition of these contextuality scenarios yields a single contextuality scenario corresponding to the experimental design. In this article, we illustrated two Bell scenario designs based on whether the "no signalling" condition holds. If this condition does hold, then the Foulis-Randall (FR) product can be used to define the composition. However, when signalling is permitted, means other than the FR product need to be developed. This is an open question which is particularly relevant to psychology experiments where signalling appears to be pervasive. In this regard, recent work on signalling in Bell scenarios may provide a useful basis for further development (Brask and Chaves, 2017). For example, Brask and Chaves (2017) studies relaxations of the "no signalling" condition where different forms of communication are allowed. The P-program depicted in Figure 12 modelled one such condition in which outcomes can be uni-directionally communicated between the two components of the assumed model. Investigating contextuality in the presence of signalling is an important issue for cognitive science and related areas. Perhaps surprisingly it is an issue that has received scant attention to date (Dzhafarov and Kujala, 2014b). When signalling is not present, it would be interesting to investigate how variations of multipartite composition of contextuality being investigated in physics may inspire new experimental designs outside of physics (Sainz and Wolfe, 2017).
Once a contextuality scenario has been constructed for the P-program, "strong contextuality" occurs when it is not possible to construct a probabilistic model on the underlying hypergraph. If a probabilistic model on the hypergraph is possible, then the random variables are independent of the measurement contexts.
The motivation for demarcating the problem into acyclic vs. cyclic cases is related to efficiency: The number of variables at the schema level is likely to be much smaller than the number of underlying events, especially when one considers larger scale experiments involving numerous random variables. This is not withstanding the fact that determining whether there is a global model turns out to be tractable. Stated more formally, given a contextuality scenario X, a linear progam can determine whether strong contextuality holds. (See Proposition 8.1.1 in (Acin et al., 2015).) This theoretical result echoes linear programming solutions which have been found for contextual semantics based on sheaf theory  and selective influence (Dzhafarov and Kujala, 2012).
One of the advantages of the hypergraph semantics of contextuality scenarios is that they are general enough to allow contextuality to be investigated in a variety of experimental settings. In the next section we show how contextuality could be investigated in an information fusion setting.

The use of P-programs for investigating contextuality in information fusion
Information fusion refers to the problem of making a judgement about an entity, situation, or event by combining data from multiple sources which are simultaneously presented to a human subject. For example, one source might be an image and another might be a social media post. Fusion allows a much better judgment to be made because it is based on multiple sources of evidence. However, the sources may involve uncertainty, for example, the human subject may not trust the source of a social media post, or the image may appear manipulated. As a consequence, a decision of trust may be contextual because a random variable T modelling trust may have different functional identities depending on the source stimulus.
Let us now sketch how a P-program could be developed to investigate whether trust is contextual. Firstly, imagine that empirical data is collected from human subjects in an experiment. For example, subjects could be simultaneously presented with two visual stimuli as is shown in Figure 14. The left hand stimulus purports to be a image of a typhoon hitting the Phillipines sourced from an obscure Asian media site. The right hand stimulus is sourced from Twitter where the language is unfamiliar (Japanese), but the graphic seems to depict a typhoon tracking towards the Phillipines. The subject must decide if they trust whether the stimuli depict the same event. Random variables affecting the decision of trust could be defined as follows: • Variables relating to the image: I 1 (e.g., "Do you trust that the image does correspond to the situation described by the text?"), I 2 (e.g., "Does the image look fake or manipulated in any way?") • Variables related to the tweet: Credibility S 1 (e.g., "Do you trust the source of the tweet to be credible?"), S 2 (e.g., "Do you trust that the tweet corresponds to the situation depicted in the image?").
These four variables allow for an experiment in which one variable of each stimulus is measured, thus imply four measurement contexts based on the the following pairs of variables: {I 1 , S 1 },{I 1 , S 2 },{I 2 , S 1 } and {I 2 , S 2 }. A between subjects design allows experimental data to be collected in each experimental context meaning a human subject is exposed to only one measurement context in order to counter learning effects. The corresponding P-program would therefore include four scopes corresponding to these measurement contexts and each scope would return the corresponding partial model based on the data collected in that measurement context. These four partial models correspond to the pairwise distributions: p(I 1 , S 1 ), p(I 1 , S 2 ), p(I 2 , S 1 ) and p(I 2 , S 2 ). As this program involves a cyclic schema, the situation is similar to that depicted in Figure 9. Therefore, measurement contexts would be defined around observations of individual variables I 1 , I 2 , S 1 , S 2 using a signalling Bell scenario design. As subjects are processing both stimuli simultaneously, it raises the possibility of signalling between the left stimulus (component A) and the right stimulus (component B).

Summary and Future Directions
The aim of this article to contribute the foundations of a probabilistic programming language that allows exploration of contextuality in wide range of applications relevant to cognitive science and artificial intelligence. The core idea is that probabilistic models are specified as a program with associated semantics which are sensitive to contextuality. The programs feature specific syntactic scopes to specify experimental conditions under which a phenomenon is being examined. Random variables are declared local to a scope, and hence are not visible to other variables. In this way, random variables can be safely overloaded which is convenient for developing models whilst the programming semantics, not the modeller, keeps track of the whether functional identities of the random variables are being preserved.
Hypergraphs were proposed as an underlying structure to specify contextually sensitive program semantics. Firstly, a hypergraph approach developed in relational database theory was used to determine whether the schema of the partial probabilistic models is acyclic. If so, the hypergraph is exploited to form a join tree which can compute a single model such that the partial models can be retrieved by appropriately marginalizing this model. In this case, the phenomenon is non-contextual. When the schema is cyclic, the phenomenon may or may not be contextual. For the cyclic case a hypergraph called a "contextuality scenario" is formed. "Strong contextuality" occurs when it is not possible to construct a probabilistic model on the hypergraph. If it is possible, then each such model is a candidate global model and the phenomenon is non-contextual. Further research could be directed at refining the semantics to admit different types of contextuality (Abramsky and Brandenburger, 2011;Acin et al., 2015), as well as experimental designs based on different variations of signalling (Brask and Chaves, 2017;Curchod et al., 2017).
Just like higher level programming languages, such as functional programming, provided a convenient means for harnessing the power of the lambdacalculus, P-programs aim to advance the understanding of contextuality, by providing a convenient means for harnessing the power of contextual semantics. As P-programs are algorithmic, future work could provide syntax to specify the temporal flow of actions using control structures akin to those used in high level programming languages. This feature allows measurements with some causal structure, which is an important topic in cognitive psychology where Bayesian models are often used.
Finally, the overarching aim of this article is to raise awareness of contextuality beyond quantum physics and to contribute formal methods to detect its presence in the form of a convenient programming language.