{'fraction_non_alphanumeric': 0.055501128764322526, 'fraction_numerical': 0.011585807385952208, 'mean_word_length': 4.141918528252299, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Categorical Aspects of Parameter Learning

Parameter learning is the technique for obtaining the probabilistic parameters in conditional probability tables in Bayesian networks from tables with (observed) data --- where it is assumed that the underlying graphical structure is known. There are basically two ways of doing so, referred to as maximal likelihood estimation (MLE) and as Bayesian learning. This paper provides a categorical analysis of these two techniques and describes them in terms of basic properties of the multiset monad M, the distribution monad D and the Giry monad G. In essence, learning is about the reltionships between multisets (used for counting) on the one hand and probability distributions on the other. These relationsips will be described as suitable natural transformations.


Introduction
Bayesian networks are graphical models for efficiently organising probabilistic information [1,2,7,17,18,20]. These models can be used for probabilistic reasoning (inference), where the probability of an observation is inferred from certain evidence. These techniques are extremely useful, for instance in a medical setting, where symptoms and measurements can be used as evidence, and the inferred probability can help a doctor reach a decision.
A basic question is how to obtain accurate Bayesian networks. This question involves two parts: how to determine the underlying graph structure, and how to obtain the probabilities in the conditional probability tables (CPTs) of the network. The first part is called structure learning, and the second part is called parameter learning. Here we concentrate on the latter, especially for discrete probability distributions.
One way of obtaining the parameters of Bayesian network is to learn them from experts. However, it is more efficient and cheaper to learn the parameters from data, if available. The data is typically organised in (very large) tables, with information of the form: so many patients had medicine A and had these symptoms and so many patients got medicine B and showed such symptoms, etc. Below we shall describe such tables, say with n dimensions, as n-ary multisets in M(X 1 × · · · × X n ), where M is the multiset monad on the category of sets.
A multiset is a 'set' in which elements may occur multiple times. We shall write such multisets as formal combinations of the form 3|a + 5|b + 2|c . This expresses that we have 3 occurrences of the element a, 5 times b, and 2 times c. Via normalisation we can turn such a multiset into a probability distribution, namely 3 10 |a + 5 10 |b + 2 10 |c . Now the parameters add up to one, via division by the sum of all occurrences. Below we shall describe this operation as a natural transformation of the form M * ⇒ D, where M * is the (sub)monad of non-empty multisets, and where D is the discrete probability distribution monad. Various properties of this natural transformation are identified, for instance with respect to marginalisation and disintegration. These properties are relevant for using this learning method with repect to a graph structure.
The above learning technique extracts probability distributions directly from the data, reformulated here via the transformation M * ⇒ D. It is called maximal likelihood estimation (MLE), see e.g. [7,Ch.17], [18, §17.1] or [17, §6.1.1]. We like to see it as 'frequentist' learning, since it is based on counting and frequencies.
There is a more sophisticated form of learning called Bayesian learning, see [7,Ch.18], [18, §17.3] or [17, §6.1.2]. It is a form of higher order learning, where one does not immediately obtain the probability distribution in D(X), for a finite set X, but one obtains a distribution over D(X). The latter is a continuous distribution, defined on a simplex, and thus involves the Giry monad G on measurable spaces. Here we show how to reformulate Bayesian learning in terms of another transformation, of the form M ⊛ ⇒ GD, where M ⊛ is the (sub)monad of multisets in which each element occurs at least once. The transformation is given by Dirichlet distributions. We show how familiar properties of Dirichlet distributions translate into categorical properties.
This paper thus gives a novel, snappy, categorical perspective on parameter learning in terms of two natural transformations: These transformations capture fundamental relationships between the multiset monad M on the one hand and probability distribution monads D and G on the other hand. In the practice of Bayesian networks, the differences between the frequentist and Bayesian learning methods are substantial, for instance wrt. to incorporating priors, variance, or zero counts, see e.g. [18, §17.3]. However, these differences are not relevant in our abstract characterisations. The topic of parameter learning is textbook material. The contribution of this paper lies in a systematic, categorical reformulation that offers a novel perspective -not only in terms of natural transformation as above, but also via conditioning, see Section 7 -that may lead to a deeper understanding and to new connections. This reformulation involves a level of mathematical precision that may be useful for (the semantics of) probabilistic programming languages.

Preliminaries on tables and distributions
This section will elaborate a simple example in order to provide background information about the setting in which parameter learning is used.
Consider the table (1) below where we have combined numeric information about blood pressure (either high H or low L) and certain medicines (either type 1 or type 2 or no medicine, indicated as 0). There is data about 100 study participants: no medicine medicine 1 medicine 2 totals (1) We consider several ways to 'learn' from this table.
We can normalise this multiset. It yields a joint probability distribution, which we call ω, for later reference: Such a distribution, directly derived from a table, is sometimes called an empirical distribution [7].
(2) The first and second marginals M 1 (ω) and M 2 (ω) of this joint probability distribution ω capture the blood pressure probabilities and the medicine probabilities separately, as: M 1 (ω) = 0.7|H +0.3|L and M 2 (ω) = 0.15|0 +0.45|1 + 0.4|2 . These marginal distributions can also be obtained directly from the above table (1), via the normalisation of last column and last row. This fact looks like a triviality, but involves a naturality property (see Lemma 2 below).
(3) Next we wish to use the above table (1) to learn the parameters (table entries) for the simple Bayesian network on the right. We then need to fill in the associated conditional probability tables. These entries are obtained from the last column in Table 1, for the initial blood distribution 0.7|H + 0.3|L , and from the two rows in the table; the latter yield two distributions for medicine usage, via normalisation.
(4) In the categorical look at Bayesian networks (see e.g. [15,16]) these conditional probability tables correspond to channels: Kleisli maps for the distribution monad D. In the above case, the channel c : {H, T } → {0, 1, 2} corresponding to the medicine table in the previous point is: We can then recover the second marginal M 2 (ω) as state transformation c ≫ M 1 (ω), see later for details.
(5) Given a joint distribution P (x, y) there is a standard way to extract a channel P (y | x) by taking conditional probabilities. This process is often called disintegration, and is studied systematically in [5] (and in many other places).
If we disintegrate the above distribution ω on the product {H, T } × {0, 1, 2} we obtain as channel {H, T } → {0, 1, 2} precisely the map c from the previous point -obtained in point (3) directly via the Table (1). This is a highly relevant property, which essentially means that (this kind of) learning can be done locally -see Proposition 4 below.
This example illustrates how probabilistic information can be extracted from a table with numeric data -in a frequentist manner -essentially by counting. This process will be analysed from a systematic categorical perspective in Section 4 below. The resulting structure will be useful in the subsequent more advanced form of Bayesian learning, see Section 6, where continuous (Dirichlet) distributions are used on the probabilistic parameters r i ∈ [0, 1] in convex combinations i r i |i . But first we need to be more explicit about the basic notions and notations that we use in our analysis.

Prerequisites on multisets and discrete probability
Categorically, (finite) multisets can be captured via a the multiset monad M on the category Sets. For a set X there is a new set M(X) = {φ : X → N | supp(φ) is finite} of multisets of X. The support supp(φ) is of a multiset φ is the subset supp(φ) = {x ∈ X | φ(x) = 0} of its inhabitants. We often write φ ∈ M(X) as formal finite sum φ = i α i |x i , with support supp(φ) = {x 1 , . . . , x n } and φ(x i ) = α i ∈ N telling how often x i ∈ X occurs in the multiset φ. The ket notation | − is meaningless syntactic sugar.
Each function h : X → Y gives rise to a function M(h) : M(X) → M(Y ) between the corresponding collections of multisets. One defines M(h) as: We do not need the monad structure of M in this paper. But functoriality -the fact that M not only acts on sets but also on maps between themplays an important role. For instance, the column and row of totals in Table ( . Discrete probability distributions -also called multinomials or categorical distributions -can be seen as special kinds of multisets, not with natural numbers as multiplicities, but with probabilities in [0, 1], with the additional requirement that these probabilities add up to one (and thus form what is called a convex combination). We write D(X) for the set of such discrete probability distributions on X. It is defined as a set of probability mass functions: An element of D(X) is often simply called a distribution (or also a state) and is written as formal convex (2). For a projection For more information about the monads M and D we refer to [12].
We often identify a natural number n with the n-element set {1, 2, . . . , n}. In this way we write Nat for the category with natural numbers n as objects and with functions n → m between them. Thus there is a full and faithful functor Nat ֒→ Sets. This category Nat has finite products, with final object 1 and binary product n × m given by multiplication of numbers.
We mostly apply the above functors M, D to n ∈ Nat, as sets. Then: We shall often (implicitly) use this isomorphism. So why use special notation M(n) for N n ? Because M is a functor and its action M(h) on a function h is most useful. -D(n + 1) can be identified with the n-simplex of r 0 , . . . , r n ∈ [0, 1] with Having seen the basics of M and D, we consider the following variations.
1. We write M * (X) ֒→ M(X) for the subset of non-empty multisets, that is, of multisets with non-empty support. More explicitly, M * (X) contains those multisets i α i |x i with α i > 0 for some index i; alternatively, the sum i α i is non-zero. Distributions have non-empty support by definition. 2. We further write M ⊛ (X) ֒→ M * (X) for the set of multisets i α i |x i with 'full support', that is with supp(φ) = X. This means that X = {x 1 , . . . , x n } and α i > 0 for all i. Using M ⊛ (X) only makes sense for finite sets X. We write D ⊛ (X) ֒→ D(X) for the subset of distribitions with full support. Functoriality is a bit subtle for M ⊛ and D ⊛ . The descriptions M(h) only makes sense for surjective functions h.
From a joint distribution ω ∈ D(X ×Y ) one can extract a channel c : X → Y , We shall use disintegration as a partial function dis : For a distribution ω ∈ D(X) on a set X and a (fuzzy) predicate p : X → [0, 1] on the same set X we write ω |= p for the validity (or expected value) of p in ω. It is the number in [0, 1] defined as x∈X ω(x) · p(x). In case this validity is nonzero, we write ω| p ∈ D(X) for the conditioned distribution, updated with predicate p. It is defined as ω|=p . For more details, see [15,16,14,11].

Frequentist learning by counting
As mentioned in the introduction, maximal likelihood estimation (MLE) is one kind of parameter learning, see e.g. [18,7,17]. We reframe it here as frequentist learning. Our categorical reformulation for discrete probability distributions (multinomials) uses the non-empty multiset functor M * and the distribution functor D from Section 3. It turns out that the process of learning-by-counting involves some basic categorical structure: it is a monoidal natural transformation, that can be applied locally.
Definition 1. For each n ∈ N we define (discrete) maximal likelihood estimation as a function ℓ n : M * (n) → D(n), determined by: The map ℓ turns numbers α i ∈ N of occurrences of data items i into a distribution, essentially by normalisation, as we have seen earlier for table (1). The distribution (5) yields a maximum for a likelihood function on distributions, see Remark 3 (2) below. Here we are interested in its categorical properties.
In particular, maximal likelihood estimation ℓ commutes with marginalisations M * (π i ) and D(π i ), obtained via projections π i : n 1 × n 2 → n i , see the naturality diagram, above on the right.
In Section 2 we mentioned that one can extract a conditional probability table (or channel) either from the table of data, or from the associated probability distribution. We are going to make this precise next. This requires some additional notation. For multisets there is an isomorphism: This row function captures disintegration for multisets. It is much simpler than for distributions, since there is no normalisation involved.
In the current situation we need to adapt the type of this row function because we wish to exclude empty multisets. Therefor we use ad hoc notation M * 2 (n×m) for the collection of multisets φ = ij α ij |ij for which M(π 1 )(φ)(i) = j α ij = 0 for each i ∈ n. Then M * 2 (n × m) ⊆ M * (n × m). We can then restrict the row-extraction function to row : M * 2 (n × m) → M * (m) n . We will next show how this row-extraction gives a link with disintegration.

Proposition 4. Disintegrations for multisets and for distributions commute with maximal likelihood estimation ℓ, as in:
Proof. We simply compute: Table (1) in Section 2 can be described 1 as an element of M * 2 (2 × 3), from which we have shown that we can extract a channel 2 → 3. The above result says that it does not matter if we form the corresponding joint distribution first and then disintegrate, or if we extract the CPT/channel directly from the table. This sometimes called a 'decomposition property' that allows us to reduce learning of a CPT/channel to a set of local learning problems. The 'local' adjective means: by ℓ m , under the exponent n, in the diagram in Proposition 4. An alternative and equivalent way to express the diagram in Proposition 4 is as equation: Thus, the following (categorical) picture emerges. Bayesian networks can be seen as graphs in the Kleisli category Kℓ(D) of the distribution monad D, see esp. [8,Chap. 4] and also [15,16]. If we write G for the underlying graph of a Bayesian network, its conditional probability tables may be described either by: Since nodes of Bayesian networks typically have finite sets as domains, we can restrict to the full subcategory Kℓ N (D) ֒→ Kℓ(D) with n ∈ N as objects.
Interestingly, a frequency table as in (1) may be reorganised similarly as a graph homomorphism G → U Kℓ N (M) , for the multiset monad M. This reorganisation happens via the above 'row' function and marginalisation.
We have seen in Remark 3 (1) that the ℓ n : M * (n) → D(n) do not form a map of monads, and hence do not produce a functor Kℓ N (M * ) → Kℓ N (M). But these ℓ n do form a graph homomorphism between the underlying graphs. Hence the passage from a frequency table to a Bayesian network (with underlying graph G) can be described as a composite of graph homomorphisms: We do not elaborate the following observation since it is not used any further. This result says that likelihood estimation yields the same outcome on separate tables as on a parallel combination of these tables. It is mostly of categorical relevance. In a learning scenario there is no point in first combining two separate tables into one, and then learning from the combination.

Prerequisites on continuous probability
Frequentist learning can be done via discrete probability distributions, but Bayesian learning requires continuous probability distributions -in particular in the form of Dirichlet distributions on probabilistic parameters in [0, 1] for the multinomial case. Categorically, such continuous distributions are captured via the Giry monad G on the category of measurable spaces [10,19,14]. Here we shall only use measurable spaces which are a bounded subset of R n , for some n, with their standard measure. Hence we do not need measure theory in full generality.
We recall that on a measurable space X = (X, Σ X ), with measurable subsets Σ X ⊆ P(X), the Giry monad G is defined as: Thus, G(f )(ω) is the 'image' measure, also called the 'push forward' measure. As mentioned, in the current context the measurable space X is typically a (bounded) subset X ⊆ R n for some n. The probability measures in G(X) that we consider are given by a 'probability density function' (abbreviated as 'pdf'). Such a pdf is a function f : X → R ≥0 with f (x) dx = 1. We write PDF(X) for the set of pdf's on X ⊆ R n , for some n.
For a measurable subset M ⊆ X we write M f (x) dx or x∈M f (x) dx for the integral 1 M (x) · f (x) dx, where 1 M : X → [0, 1] is the indicator function for M ; it is 1 on x ∈ M and 0 on x ∈ M . In this way we can define a function: Disintegration for continuous joint distributions is much more difficult than for discrete distributions. However, in our current setting (see also [6,5]), where we restrict ourselves to distributions ω ∈ G(X × Y ) given by a pdf, say ω = I(f ) = f , for f : X × Y → R ≥0 , there is a formula to obtain dis(ω) ∈ G(Y ) X , namely: dis For a continuous distribution ω ∈ G(X) on a measurable space X and a measurable function q : X → [0, 1] we write ω |= q for the 'continuous' validity value in [0, 1], obtained via Lebesgue integration q dφ. If the measure φ is given by a pdf f , as in φ = I(f ) = f , then q dφ equals q(x) · f (x) dx. Also in the continuous case we can update a distribution ω ∈ G(X) to ω| p ∈ G(X) via the definition: 6 Bayesian learning In Section 4 we have uncovered some basic categorical structure in frequentist parameter estimation. Our next challenge is to see if we can find similar structure for Bayesian parameter estimation, where parameters (for discrete probability distributions, or multinomials) are obtained via successive Bayesian updates. This will involve (continuous) Dirichlet distributions over the probabilistic parameters r i ∈ [0, 1] of distributions i r i |x i . For Dirichlet distributions it is required that each of these r i ∈ [0, 1] is non-zero. We shall write D ⊛ (n) ⊆ D(n) ⊆ R n for the subset of (r 1 , . . . , r n ) with r i > 0 for each i ∈ n. We shall study Dirichlet distributions via their pdf's written as d n in: given by: The variables α i ∈ N >0 are sometimes called hyperparameters, in contrast to the 'ordinary' variables x i ∈ (0, 1]. We use these Dirichlet pdf's d n to define what we call the Dirichlet distribution functions Dir n in: Hence Dir n (α)(M ) = I d n (α) = M d n (α)(x) dx, see (9). Our main aim in this section is to prove that the maps Dir n : M ⊛ (n) → G D ⊛ (n) are natural in n -in analogy with naturality for ℓ n : M * (n) → D(n) in Lemma 2 -but only for surjective functions. We proceed via a functorial reformulation of the aggregation property of the Dirichlet functions.
There are some basic facts that we need about these d n , see [3] for details.
2. The fraction in (12) works as a normalisation factor, and satisfies: This ensures that d n (α) is a pdf. 3. For each α ∈ M ⊛ (n) and i ∈ n one has: The first equation follows easily from the previous two points. The second equation establishes a (standard) link between maximal likelihood estimation ℓ n and Dirichlet pdf's d n .
Whereas results about maximal likelihood estimation in Section 4 are relatively easy, things are mathematically a lot more challenging now. We need the socalled aggregation property of the Dirichlet pdf's d n , from [9]. Our contribution is a reformulation this property -not the property itself -in functorial form, starting from a formulation in the first point below that is close to the orginal.
Once we know the basic aggregation form (16) we can generalise by using the funtoriality of M and D. It gives a succint functorial reformulation of the aggregation property of the Dirichlet functions [9]. Lemma 7. Let h : n ։ m be a surjective function. For α ∈ M ⊛ (n) and x ∈ D ⊛ (m) one has: Moreover, for a measurable subset N ⊆ D ⊛ (m): We thus arrive at the main result of this section.
Lemma 8. The Dirichlet maps Dir n are natural for surjective functions, that is, for a surjective function h : n ։ m the following diagram commutes.

A logical perspective on learning
This section examines frequentist and Bayesian learning from a logical perspective, using the notions of validity |= and conditioning. Equation (15) expresses that the expected value of data element i under the Bayesian interpretation coincides with the probability of i under a frequentist interpretation. We shall extend this correspondence in terms of predicates and their validity |=, as explained at the end of Section 3 and 5.
For a predicate p : n → [0, 1] we can write (−) |= p : D(n) → [0, 1] for the map ω → ω |= p. It is in fact the Kleisli extension of p. This map p := (−) |= p is now a predicate on D(n) from the continuous perspective. Hence we can look at p's validity. Below we relate validity |= for the discrete probability distribution ℓ n (α) to validity |= for the continuous probability distribution Dir n (α) via an adaptation of the predicate, from p to p. It provides a fancy extension of (15). Proof. We unpack the definitions and use Equation (15) in a crucial manner: The Bayesian approach to learning can handle additional data via conditioning. This will be made precise below, using a logical formulation that is characteristic for conjugate priors, see [13]. For numbers n and i ∈ n we write 1 {i} : n → [0, 1] for the singleton predicate that is 1 on i ∈ n and 0 elsewhere. Theorem 10. For α = (α 1 , . . . , α n ) ∈ M ⊛ (n) and i ∈ n write α ++ i for the sequence (α 1 , . . . , α i +1, . . . , α n ) in which the i-th entry is is incremented by one. A Dirichlet distribution updated with a point i observation is the same as this distribution with the i-th hyperparameter incremented by one: Proof. Write α = i α i . By Proposition 9 we have: Thus, for a measurable subset M ⊆ D ⊛ (n), This update property does not work for the frequentist approach: update of an existing distribution with point evidence trivialises the distribution: ℓ n (α) 1 {i} = j ℓ n (α)(j) · 1 {i} (j) ℓ n (α) |= 1 {i} j = ℓ n (α)(i) ℓ n (α)(i) i = 1|i .

Conclusions
Specific natural transformations M * ⇒ D and M ⊛ ⇒ GD have been identified as the crucial ways of going from data to distributions in parameter learning. This categorical approach may shed light on non-trivial applications of learning, for instance in topic modelling via latent Dirichlet allocation [4,21].