{'fraction_non_alphanumeric': 0.04959270670662221, 'fraction_numerical': 0.008103659308656565, 'mean_word_length': 4.222393652193079, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Diagrammatic Differentiation for Quantum Machine Learning

We introduce diagrammatic differentiation for tensor calculus by generalising the dual number construction from rigs to monoidal categories. Applying this to ZX diagrams, we show how to calculate diagrammatically the gradient of a linear map with respect to a phase parameter. For diagrams of parametrised quantum circuits, we get the well-known parameter-shift rule at the basis of many variational quantum algorithms. We then extend our method to the automatic differentation of hybrid classical-quantum circuits, using diagrams with bubbles to encode arbitrary non-linear operators. Moreover, diagrammatic differentiation comes with an open-source implementation in DisCoPy, the Python library for monoidal categories. Diagrammatic gradients of classical-quantum circuits can then be simplified using the PyZX library and executed on quantum hardware via the tket compiler. This opens the door to many practical applications harnessing both the structure of string diagrams and the computational power of quantum machine learning.


Introduction
String diagrams are a graphical language introduced by Penrose [33] to manipulate tensor expressions: wires represent vector spaces, nodes represent multi-linear maps between them. In [34], these diagrams are used to describe the geometry of space-time and an extra piece of notation is introduced: the covariant derivative is represented as a bubble around the tensor to be differentiated. Joyal and Street [25,26] characterised string diagrams as the arrows of free monoidal categories, however their geometry of tensor calculus makes no mention of differential calculus, it only deals with composition and tensor.
In categorical quantum mechanics [1] string diagrams are used to axiomatise quantum theory in terms of dagger compact-closed categories. This culminated in the ZX-calculus [9], a graphical language that provides a complete set of rules for qubit quantum computing [24,20]. ZX diagrams have recently been used for state-of-the-art quantum circuit optimisation [29,17,14], compilation [12,16], extraction [2] and error correction [5,19]. In recent work, ZX diagrams have been used to study quantum machine learning [40,41] and its application to quantum natural language processing [30,8].
In this work, we introduce diagrammatic differentiation: a graphical notation for manipulating tensor derivatives. On the theoretical side, we generalise the dual number construction (discussed in section 1) from rigs to monoidal categories (section 2). We then apply this construction to the category of ZX diagrams (section 3) and of quantum circuits (section 4). In section 5 we give a formal definition of diagrams with bubbles and their gradient with the chain rule. We use this to differentiate quantum circuits with neural networks as classical post-processing. The theory comes with an implementation in DisCoPy [15], the Python library for monoidal categories. The gradients of classical-quantum circuits can then be simplified using the PyZX library [28] and compiled on quantum hardware via the tket compiler [36].

Related work
The same bubble notation for vector calculus is proposed in [27], but they have mainly pedagogical motivations and restrict themselves to the case of three-dimensional Euclidean space. To the best of our knowledge, our definition is the first formal account of string diagrams with bubbles for tensor derivatives.
Differential categories [4] have been introduced to axiomatise the notion of derivative. More recently reverse derivative categories [7] generalised the notion of back-propagation, they have been proposed as a categorical foundation for gradient-based learning [13]. These frameworks all define the derivative of a morphism with respect to its domain. In our setup however, we define the derivative of parametrised morphism with respect to parameters that are in some sense external to the category. Investigating the relationship between these two definitions is left to future work.

Dual numbers
Dual numbers were first introduced by Clifford in 1873 [6]. Given a commutative rig (i.e. a riNg without Negatives) S, the rig of dual numbers D[S] extends S by adjoining a new element such that 2 = 0. Concretely, elements of D[S] are formal sums + where and are scalars in S. We write 0 , 1 : D[S] → S for the projection on the real and epsilon component respectively. Addition and multiplication of dual numbers are given by: A related notion is that of differential rig: a rig S equipped with a derivation, i.e. a map : S → S which preserves sums and satisfies the Leibniz product rule ( × ) = × ( ) + ( ) × for all , ∈ S. An equivalent condition is that the map ↦ → + ( ) is a homomorphism of rigs S → D[S]. The correspondance also works the other way around: given a homorphism : S → D[S] such that We can use equations 1, 2 and 3 to derive the usual rules for gradients in terms of dual numbers. For the identity function we have id( + ) = id( ) + , i.e. id = 1. For the constant functions we have ( + ) = ( ) + 0 , i.e. = 0. For addition, multiplication and composition of functions, we can derive the following linearity, product and chain rules: This generalises to smooth functions R → R , where the partial derivative is a derivation for each < . The functions F 2 → F 2 on the two-element field F 2 with elementwise XOR as sum and conjunction as product also forms a differential rig. The partial derivative is given by ( ) ( ì) = (ì [ ↦ →0] ) ⊕ ( ì [ ↦ →1] ). Intuitively, the F 2 gradient ( ì) ∈ F 2 encodes which coordinates of ( ì) actually depend on the input . An example of differential rig that isn't also a ring is given by the set N[ ] of polynomials with natural number coefficients, again each partial derivative is a derivation. A more exotic example is the rig of Boolean functions with elementwise disjunction as sum and conjunction as product. Boolean functions B → B can be represented as tuples of propositional formulae over variables. The partial derivative for < is defined by induction over the formulae: for variables we have = , for constants 0 = 1 = 0 and for negation ¬ = ¬ . The derivative of disjunctions and conjunctions are given by the linerarity and product rules. Equivalently, the gradient of a propositional formula can be given by Concretely, a model satisfies if and only if it satisfies ↔ : the derivative is true when the variable and the formula are positively correlated. Substituting with its negation, we get that a model satisfies [ ↦ →¬ ] if and only if it satisfies ↔ ¬ , i.e. iff variable and formula are anti-correlated. Note that although B and F 2 are isomorphic as sets, they are distinct rigs. Their derivations are related however by F 2 ↦ → B ∨ B [ ↦ →¬ ] for : B → B the formula corresponding to the function : F 2 → F 2 . That is, a Boolean function depends on an input variable precisely when either the corresponding formula is positively correlated or anti-correlated.
Dual numbers are a fundamental tool for automatic differentiation [23], i.e. they allow to compute the derivative of a function automatically from its definition. The key idea is that given a definition of : S → S as a composition of elementary functions, we can compute ( ) ( ) by evaluating ( + ) and projecting on the epsilon component.

Dual diagrams
Our main technical contribution is to generalise derivations from rigs to monoidal categories with sums. Applying this to free monoidal categories, where the arrows are string diagrams, we say a derivation is diagrammatic when it commutes with the interpretation of the diagrams. We take two different flavours of the ZX-calculus as our main examples.
Let (C, ⊗, 1) be a monoidal category with sums, i.e. it has commutative monoids on each homset (+) : , C( , ) × C( , ) → C( , ) with unit 0 ∈ , C( , ) such that composition and tensor distribute over the sum. Note that a one-object monoidal category with sums is simply a rig. Our motivating example is the category Mat S with natural numbers as objects and matrices valued in a commutative rig S as arrows, with matrix multiplication as composition, Kronecker product as tensor and entrywise sum. We define the category D[C] by adjoining a scalar (i.e. an endomorphism of the monoidal unit) such that ⊗ = 0 . Concretely, the objects of D[C] are the same as those of C, the arrows are given by formal sums + of parallel arrows , ∈ C. Composition and tensor are both given by the product rule: We say that a unary operator on homsets : is a derivation whenever it satisfies the product rules for both composition Proof. The isomorphism is given by

Proposition 2.2. Derivations on Mat S are in one-to-one correspondance with derivations on S.
Proof. A derivation on Mat S is uniquely determined by its action on scalars in S. Conversely, applying a derivation : S → S entrywise on matrices yields a derivation on Mat S .
Fix a monoidal signature Σ with objects Σ 0 and boxes Σ 1 . Let C Σ be the free monoidal category it generates: the objects are types, i.e. lists of generating objects = 1 , . . . , ∈ Σ ★ 0 , the arrows are string diagrams with boxes in Σ 1 . Let C + Σ be the free monoidal category with sums: the objects are also given by types, the arrows are formal sums, i.e. bags , of string diagrams. We assume our diagrams are interpreted as matrices, i.e. we fix a sum-preserving monoidal functor [[−]] : C + Σ → Mat S for S a commutative rig with a derivation : S → S. Our main two examples are the standard ZX-calculus with smooth functions R → R as phases and the algebraic ZX-calculus over S, introduced in [39].
Applying the dual number construction to C + Σ , we get the category of dual diagrams D[C + Σ ] which is where diagrammatic differentiation happens. By the universal property of C + Σ , every derivation : is uniquely determined by its image on the generating boxes in Σ 1 . Intuitively, if we're given the derivative for each box, we can compute the derivative for every sum of diagram using the product rule. We say that the interpretation

the interpretation of the gradient [[ ]] coincides with the gradient of the interpretation [[ ]]
for all sums of diagrams ∈ C + Σ . We depict the gradient as a bubble surrounding the diagram , we introduce bubbles formally in section 5. Once translated to string diagrams, the axioms for derivations on monoidal categories with sums become:

Differentiating ZX
This section applies the dual number construction to the diagrams of the ZX-calculus.
We write ZX + for the category of formal sums of parametrised ZX diagrams.

Remark 3.2. Note that we've scaled the standard interpretation of the green spider by a global phase to match the usual definition of rotation gates in quantum circuits.
Remark 3.3. For = 0 we get ZX 0 = ZX the ZX-calculus with no parameters. By currying, any ZX diagram ∈ ZX can be seen as a function :  Note that there is no diagrammatic differentiation for the ZX-calculus with smooth maps as phases, even when restricted to bounded functions. Take for example : R → R with ( ) = sin 2 , it is smooth and bounded by 1 but its derivative is unbounded. Thus, from lemma 3.4 we know it cannot be represented as a scalar diagram in ZX 1 : there can be no diagrammatic differentiation : ZX 1 → D[ZX 1 ]. In such cases, we can always extend the signature by adjoining a new box for each derivative. The issue of being able to represent arbitrary scalars disappears if we work with the algebraic ZX-calculus instead. Furthermore, we can generalise from S = R → C to any commutative rig.
Definition 3.8. The diagrams of the algebraic ZX-calculus over a commutative rig S form a category ZX S = C Σ where the signature Σ is given in [39, Table 2] and the interpretation is given in [39, §6]. In particular, there is a green square , ( ) ∈ Σ 1 for each ∈ and , ∈ N with [[ , ( )]] = |0 ⊗ 0| ⊗ + |1 ⊗ 1| ⊗ . Let ZX + S be the category of formal sums of algebraic ZX diagrams over S. can be represented by the scalar diagram 1,0 ( )|1 . In the other direction, a diagrammatic derivation on ZX + S is uniquely determined by its action on scalars 1,0 ( )|1 for ∈ S.
One application of diagrammatic differentiation is to solve differential equations between diagrams. As a first step, we apply Stone's theorem [37] on one-parameter unitary groups to the ZX-calculus.  Proof. Given the diagram for a unitary group , we compute its diagrammatic differentiation and get ℎ by pattern matching. Conversely given a self-adjoint ℎ, the diagram = exp( ℎ) is a unitary group.

Differentiating quantum circuits
In this section, we extend diagrammatic differentiation to classical-quantum circuits. These circuit diagrams have two kinds of wires for bits and qubits, and boxes for pure quantum processes, measurements and preparations. We interpret these classical-quantum circuits in terms of parametrised matrices, where the tensor product reorders the indices to keep the classical and quantum dimensions in order. Borrowing the term from Coecke and Kissinger [10], we call these matrices cq-maps. In this context, diagrammatic derivations correspond to the notion of gradient recipe for parametrised quantum gates [35]. We first give the definition of parametrised cq-maps which is at the basis of our Python implementation. The category CQMap has objects given by pairs of natural numbers Ob(CQMap ) = N × N, where the first and second element of the pair encode the classical and the quantum dimension of the system respectively. Arrows : ( , ) → ( , ) are given by × 2 → × 2 parametrised complex matrices, i.e. with entries in R → C. Composition of cq-maps is given by multiplying their underlying matrices. Tensor is given on objects by pointwise multiplication and on arrows by the following diagram in Mat R →C : In quantum mechanical terms, this corresponds to the distinction between quantum superposition and probabilistic mixing. Remark 4.1. The cq-maps we have defined here differ from [10] in two minor ways. First, we take the algebraic conjugate rather than the diagrammatic conjugate, i.e. we take ⊗ = ⊗ ≠ ⊗ . This is just a choice of convention that makes numerical computation easier. Second, our category CQMap contains matrices that have no physical interpretation, e.g. we do not ask for complete positivity. This can be fixed by considering the subcategory in the image of the interpretation functor defined below.
Take a monoidal signature Σ with one object Σ 0 = { } interpreted as a qubit, and boxes interpreted as pure quantum processes with parameters. That is, we fix a parametrised interpretation functor . We write (ZX ) for the category of classical-quantum circuits with parametrised ZX diagrams as pure processes.
Let C + (Σ) be the free monoidal category with sums, i.e. arrows are bags of circuits. Again, we want to find a diagrammatic derivation : C + (Σ) → D[C + (Σ) ] which commutes with the interpretation, i.e.

such that [[ˆ]] = [[ˆ]] = [[ ]] ⊗ [[ ]]
for all pure maps ∈ Σ 1 . Note that a diagrammatic derivation for pure processes in C + Σ does not in general lift to one for classical-quantum circuits in C Σ . Indeed, using the product rule we get [

[ ]] ⊗ [[ ]] = [[ ]] ⊗ [[ ]] + [[ ]] ⊗ [[ ]] ≠ [[ ]] ⊗ [[ ]].
Hence we need equations, called gradient recipes, to rewrite the gradient of a pure map [[ˆ]] as the pure map of a gradient [[ˆ]]. In the special case of Hermitian operators with at most two unique eigenvalues, gradient recipes are given by the parameter-shift rule. In the general case where the parameter-shift rule does not apply, gradient recipes require the introduction of an ancilla qubit. Proof. The rotation has eigenvalues ±1, hence the spiders with two legs have diagrammatic differentiation given by the parameter-shift rule: As for theorem 3.6, this extends to arbitrary-many legs using spider fusion. Example 4.5. The quantum enhanced feature spaces of [21] are parametrised classical-quantum circuits. The quantum classifier can be drawn as a diagram: where ( ì) depends on the input, ( ì ) depends on the trainable parameters and is a fixed Boolean function encoded as a linear map.

Bubbles and the chain rule
This section introduces an extension of the language of string diagrams that encodes arbitrary non-linear operators on matrices: bubbles. Previous sections already used two kinds of bubbles informally: matrix exponentials and gradients. We give a formal definition of bubbles and their gradients with the chain rule. We then use them to compute the gradient of hybrid classical-quantum circuits where the measurement results can be post-processed by any classical feed-forward neural network.
Fix a set of colours . Take a monoidal signature Σ, we construct the free monoidal category with sums and bubbles C + (Σ) , i.e. arrows are formal sums of diagrams with bubbles. We define the signature of bubbled diagrams as a union (Σ) = ∈N (Σ, ) where the signature of (≤ )-nested bubbles (Σ, ) is defined by induction: That is, we put a formal sum of diagrams ∈ C + (Σ, ) with (≤ )-nested bubbles inside a -coloured bubble and take it as a box ( ) ∈ (Σ, + 1) for diagrams with ( + 1)-nested bubbles. We say a monoidal category C has bubbles when it comes equipped with a unary operator on homsets : , C( , ) → C( , ) for each colour ∈ .
Although it makes the bureaucracy heavier, we may consider bubbles that change the domain and codomain of the diagram inside. Such a bubble is defined by two operators on objects dom , cod : Ob(C) → Ob(C) and an operator on homsets : , C( , ) → C( dom ( ), cod ( )). Example 5.1. Bubbles first appear in Penrose and Rindler [34] where they are used to encode the covariant derivative. An extra wire comes in the bubble to encode the dimension of the tangent vector. Example 5.2. The functorial boxes of Melliès [31] can be thought of as well-behaved bubbles, i.e. such that the composition of bubbles is the bubble of the composition. Indeed, a functor : C → D between two categories C and D defines a bubble on the subcategory of their coproduct C D spanned by C. Example 5.3. Bubbles appear under the name "uooh" (unary operator on homsets) in [22] where they are used to encode the sep lines of C.S.Peirce's existential graphs. Take the predicates of a first-order logic as signature, i.e. one generating object and each predicate with arity as a box with dom( ) = 1 and cod( ) = ⊗ . Add generators for spiders to encode lines of identity. Then bubbled diagrams encode first-order logic formulae, and every formula can be represented in this way. Logical deduction rules may be given entirely in terms of diagrammatic rules. The evaluation of first-order logic formulae is a bubble-preserving functor : C (Σ) → Mat B , where bubbles are interpreted as pointwise negation. Example 5.4. Take colours to be arbitrary rig-valued functions S → S, then the category of matrices Mat S has bubbles given by pointwise application. Gradient bubbles : S → S are a special case. Example 5.5. In the subcategory of square matrices, matrix exponential is an example of bubble for S = R, C. When S = B, square matrices are finite graphs and reflexive transitive closure is an example. Example 5.6. Bubbles can encode the standard non-linear operators used in machine learning. The sigmoid ( ) = 1/(1 + − ) and rectified linear unit ( ) = max(0, ) are pointwise bubbles : R → R. The softmax function : R → R takes a vector ì, applies exponential pointwise then normalises by < ì . It can be drawn as a bubble around the diagram for the vector ì. Bubbles may also depend on the labels from the dataset. Take a loss function such as the relative entropy ( ì, ì ★ ) = < ì log( ì /ì ★ ). The partially-applied loss function (−, ì ★ ) : R → R for the label ì ★ ∈ R can be drawn as a bubble around the diagram for the prediction ì ∈ R .
Bubbles compose by nesting, this defines a category of post-processes (C). The objects are pairs of objects from C, arrows ( , ) → ( , ) are -coloured bubbles such that dom ( ) = and cod ( ) = .
If we apply this to the category of matrices, (Mat S ) is the category of all matrix-valued functions. In particular, this includes any feed-forward neural networks. Indeed, take = • · · · • 1 : R → R where each layer is given by ( ì ) = ( ì + ) for the input vector ì : 1 → , the parametrised weight matrix : → and bias vector : 1 → in Mat R →R for the total number of parameters. Drawing both the layer and the activation : R → R as bubbles we get the following definition: When the bubble has a derivative , we may define the gradient of bubbled diagrams with the chain rule ( ( )) = ( ) ( ) × . In order to make sense of the multiplication, we assume that the homsets of our category C have a product on homsets which is compatible with the sum, i.e. each homset forms a rig and which commutes with the tensor, i.e. ( × ) ⊗ ( × ) = ( ⊗ ) × ( ⊗ ). The category of matrices Mat S over a rig S is an example, each homset Mat S ( , ) is a rig with entrywise sums and products. Another example is the category of diagrams with spiders on each object, where the product is given by pre/post-composition with the co/monoid structure. We get the following equation: = For scalar diagrams, spiders are empty diagrams and the equation simplifies to the usual chain rule.
Thus, we can draw both a parametrised quantum circuit and its classical post-processing as one bubbled diagram in (ZX ). By applying the product rule to the quantum circuit and the chain rule to its post-processing, we can compute a diagram for the overall gradient. This applies to parametrised quantum circuits seen as machine learning models [3], to the patterns of measurement-based quantum computing seen as ZX-diagrams [18] as well as the quantum natural language processing of [30].

Conclusion, implementation & future work
We introduced diagrammatic differentiation for tensor calculus, using bubbles to represent the partial derivative of a subdiagram. The product rule allows to compute the gradient of a diagram from the gradient of its boxes. Applying this to ZX diagrams, we showed how to compute the gradient of any linear map with respect to a phase parameter. We then extended this to quantum circuits with the parameter-shift rule and to neural networks with the chain rule.
Although this work focused on the theoretical foundations of diagrammatic differentiation, we briefly describe its implementation as part of the open-source DisCoPy library [15]. A notebook with examples is available in the documentation . The cqmap module implements classical-quantum maps as NumPy arrays [38], with SymPy [32] symbols as parameters. The two modules zx and circuit build upon monoidal, the implementation of diagrams in monoidal categories. They both come with an eval method which evaluates a diagram as a NumPy array and a grad method which returns a formal sum of diagrams given a SymPy symbol. The zx module comes with back-and-forth translations with the PyZX library [28] for automated diagram simplification. The circuit module interfaces with the tket compiler [36], allowing to execute the diagrams for circuits and their gradient on quantum hardware.
For now, we have only defined gradients of diagrams with respect to one parameter at a time. In future work, we plan to extend our definition to compute the Jacobian of a tensor with respect to a vector of variables. Other promising directions for research include the study of diagrammatic differential equations, as well as a definition of integration for diagrams.
We do not assume that products are compatible with composition, in other words C need not be rig-enriched. https://discopy.readthedocs.io/en/main/notebooks/diag-diff.html