{'fraction_non_alphanumeric': 0.03169160913682065, 'fraction_numerical': 0.004355279618218042, 'mean_word_length': 4.51033954557059, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Computing with Cognitive States

Basic experimental findings about human working memory can be described by an algebra built on high-dimensional binary states, representing information items, and two operations: multiplication for binding and addition for bundling. In contrast to common VSA algebras, bundling is not associative. Consequently bundling a sequence of items preserves their sequential ordering. The cognitive states representing a memorised list exhibit a primacy as well as a recency gradient. The typical concave-up and asymmetrically shaped serial position curve is derived as a linear combination of those gradients. Quantitative implications of the algebra are shown to agree well with empirical data from basic cognitive tasks including storage and retrieval of information in human working memory.


Introduction
Cognitive functions are established by the interplay of processes in the brain, in which patterns of neuronal activity interact and are consecutively transformed. They can be excited by some input and might finally evoke or suppress some behavioural output. Related computations are governed by simple binary units, e.g. neurons, which constitute a highly complex medium, i.e. a huge heterogeneous and irregular synaptic network. Thus, complexity is in the medium rather than in the units. Transferring this view to the question about how an algebra governing these computations should look, leads to the framework of high-dimensional computing: The algebra governing such computation is defined on a complex, i.e. high dimensional and random space, while rules for bundling or collecting information and multiplication for binding information items together are elementary binary operations.
Experimental research has revealed a number of typical findings about the functioning of memory. According to the above view, the 'cognitive algebra;' to be proposed should be able to reproduce such findings. The question therefore is: "Can the algebra reproduce the experimental data about memory, and how far can be go with this algebra alone." Thus the aim is to analyse the corresponding algebra alone, especially with regard to its congruence with empirical findings including the serial position curve.
Human Working Memory is commonly regarded as a functional subsystem of memory, whose goal is to hold and to organise information for some short period of time in order to make it available for higher cognitive processes [1]. Experiments in this field rely on the subtle construction of input data such as memory lists and produce output data such as recall probabilities or response times [8,6,12]. Among these, the most prominent finding is the serial-position curve, which shows the accuracy of item retrieval varying as a function of serial position in a memory list, averaged over a sample of participants. As observed across (probably all) immediate memory tasks, it has a concave-up shape and is asymmetric. Its particular shape depends on the particular cognitive task. such as recognition, free recall, backward and forward serial recall, or cued recall. For example in recognition and in cued (probed) recall the serial position curve shows a strong recency effect, while the primacy effect is weak. Strong primacy effects are seen in forward recall, while recency effects are strong in backward or in free recall.
To describe particular aspects of the functioning of the human working memory, models with different characteristics have been used, differing both in terms of the medium in which the information is stored and the storage operations used. Models include local code models such as REM [15], distributed models of memory such as SOB [2] and TCM [5], as well as holographic models such as TODAM [9,10] which uses high-dimensional probabilistic encoding for the holographic representation of information [13]. Holographic models gain from the properties, which are implied by high dimensionality together with randomness, see [7] for an overview about the framework of high-dimensional computing.
The holographic approach appears as a natural candidate to model the functioning of cognitive processes. Input items evoke activity patterns in the respective neural field; The fact that these representations are sparse and the consequences thereof are not explicitly considered in this note. Computation consists in transforming those patterns according to two elementary operations: The additive-like superposition realises the bundling of item information, while multiplication realises binding of items. The high-dimensional space of binary patterns together with these two operations form a highdimensional algebra governing storage and computation in this system.
Before giving an outline of the paper, a remark seems worthwhile: The aim is not to provide a full-blown model rather than to propose an elementary computational structure, an algebra, on top of which a model could be constructed. The main question is, how much of experimental findings can already be described on the basis of that algebra alone.
The outline of this paper is briefly as follows: Firstly, the state-space is defined as a high-dimensional Hamming space (eq 1) equipped with some distance on it. A similarity measure is proposed which is derived from that distance. It allows both, to judge about the familiarity of two states as well as about their distinctiveness (eq 2). Computing is by manipulating states according to two operations on that space: multiplication for binding and a not associative addition for bundling. This completes the definition of the algebra (eq 4) to be considered. Non-associativity is an essential feature of that bundling since it implies that the sum of components depends on their sequential ordering (eq 5). As a consequence, information about the order of sequentially presented list items is conserved. The corresponding left-associative sum and the right-associative sum of list items correspond to states exhibiting a recency and a primacy gradient, respectively (Fig 4). As applications basic cognitive tasks such as item recognition and probed recall are considered. The typical concave-up and asymmetrical shape of the serial-position curve is derived as a mixture of these two activation gradients (Fig 9).

The state-space
In the course of perceiving a physical item, the corresponding sensory input invokes an activity pattern in the neuronal field it is projected to. That way, each physical item can be represented by a binary pattern, in which 1 s indicate active neurons, while 0 s indicate inactive ones. Due to the size and structural complexity of the neuronal correlate, patterns are described by high-dimensional random binary vectors. These patterns are the states of the cognitive system. The state-space therefore is N > 100 is its dimension, q is the mean activity of a state, and d is some metric on X N q . The state-space is a (metric) Hamming space allowing for some similarity measure derived from the distance d. This measure should respect both: the closeness of two states as well as their distinctiveness as points in the statespace. A cosine-similarity only reveals information about closeness since it is locally defined. In a probabilistic setting, two points are the more difficult to distinguish, the less likely it is to find another state at random which is 'in between' the two. To capture this, the definition of similarity must contain global information about the state space.
Definition (Similarity). The similarity of two states having distance d from each other is Different items are represented by uncorrelated states, while similar items will be represented by similar states. κ > 1 is chosen to have highest sensitivity with respect to almost identical or near-by states.

The operations
The two operations to be defined on the state space correspond to binding and bundling. Two items are (associatively) bound to each other, if one can be retrieved by cueing with the other item. The corresponding formal operation is multiplication * , which is defined in eq 3. Binding of items happens by simultaneously activated components in the neural pattern. This similarity measure directly relates to a recall probability or accuracy of retrieval.
Bundling means collecting items by adding their respective states. Assume that two neurons X and Y converge on a third neuron Z. If both are inactive, i.e. x = y = 0, neuron Z will also be, z = 0, while if both are active, Z will be active, i.e. 1 + 1 = 1. If only X or Y is active, it depends on some threshold, whether Z is active. If the activation threshold is low, p ≈ 0, Z is likely to be active, while if if the activation threshold is high, p ≈ 1, Z will remain inactive. Addition x p is defined in eq 3. * 0 1 0 1 0 1 0 1 where ζ ∈ {0, +1} is random with P[ζ = 0] = p. This completes the definition of the algebra used to calculating with cognitive states.
In the following, its elementary properties are further investigated. What properties are already implied by this elementary algebra and how much of empirical findings can be already described by those?
Bundling preserves sequential information in the memory list Usually, bundling is realised by vector-addition [14], which is commutative and associative, so that x + (y + z) = (x + y) + z = z + (y + z), i.e. the order of components doesn't matter. That is: If addition is associative, sequential order information is lost! Observation. For 0 < p < 1, addition + p is not associative.
Note that, if p = 1, addition equals component-wise AN D, while for p = 0, addition is component-wise OR. These operations are associative.
In the following, the state resulting from left-associative addition is denoted by L, i.e. L = (x + p y) + z, while the state resulting from rightassociative addition is denoted by R = x + p (y + p z). For the sake for readability, I will write + = + p in the following, while assuming that p = 1 2 .

The states representing a memory list
L and R states can be constructed for a list of any length. Construction starts from a pre-experimental state η and proceeds by iteratively adding items to the memory states L and R according to left-associative addition and right-associative addition to the respective branch as follows: For the L-state while for the R-state After its sequential presentation, the memory list Λ = (A, B, C, . . .) is thus represented by the two states In [9] η is assumed to be empty, while in [3] it comprises a holographic collection of items and item-item associations. a, b, c, . . . are the cognitive states representing the physical list items A, B, C, . . .. These states preserve the serial order of items in the memory list in that distances change monotonously along subsequent items, see Fig. 3 d Correspondingly, both states inherit serial order in that item distances increase along R, while they decrease along L, see Fig 3. These distance gradients directly translate into activation gradients. Figure 3: Distance profiles of the two states L and R as in eq 8 ff. L has smallest distances to the most recent items, while R is closest to the early list items.

Implied activity gradients
From the concept of similarity, two other concepts can be immediately derived: activation and memory strength. The intuition is closely related to the idea of a projection. Given that the memory state M represents a memorised list, and that a cue item is presented. The cue item activates the memory state more, the more similar it is to that memory state [?]. Conversely, the more the corresponding memory element is engraved in the memory state, the more the memory state is activated by the cue state.
The activation gradient of M is the vector α M with components α M (x), where x is a state representing a list item. In terms of strength theory, α M (x) is the strength by which x is memorised in M. One might also call α M (x) the familiarity of x given M.
Consequently, the distance gradients in eq 8 ff directly translate into activity gradients, see Fig. 4. Since activation as well as strength are increasing functions of similarity and hence decreasing functions of distance, L implies a recency gradient α L , while R implies a primacy gradient α R .
Activation gradients are nowadays widely accepted to play an important role in working memory. Various mechanisms have been discussed as sources of these gradients, see [11]. In many models including TODAM, TCM and SOB, these gradients are separately modelled and superimposed on top of the model. In contrast, these gradients directly result from the bundling operation defined in eq 4 and its non-associativity: While non-associativity preserves information about serial order, right-associative addition and leftassociative addition imply the primacy and the recency gradient, respectively.

The response function for recognition and recall
After presentation of a memory list, the participant has to fulfil some task. Most cognitive tasks involve cues such as cued item recognition or cued recall, associative or serial. The answer the participant gives is the result of a decision process which depends on both, the memory state as well as the cue. The response function in recognition only depends on familiarity, while the response function in recall additionally depends on distinctiveness [9]. Thus it is reasonable to make the response function a function of activation as defined in eq 11.
Definition (Response function). The response function given a cue x facing the memory state M is an increasing function of induced similarity, e.g.
Accordingly an activation gradient directly translates into a serial position curve. Particularly, the recency effect refers to the activation gradient of the L-state, while the primacy effect corresponds to the activation gradient of the Rstate.
Experimental data indicate that the recency effect does not depend on list length and shows a slightly sigmoid curve shape, see

Application to some basic cognitive tasks
In this section some examples are presented to demonstrate how the formalism works, i.e. how to describe tasks such as cued recall in this formalism. Results are direct consequences of the algebra defined, i.e. no further assumptions are made. In the following only the R-state is concerned, i.e. states are bundled according to right-associative addition, while corresponding brackets are skipped for the sake of readability.

Repetition increases strength
It is intuitively expected that a repeated occurrence of an element in a list will increase its coding strength. This effect is indeed observed in the model. As a benchmark, consider the list Λ = (A, B, C, D, . . .), in which all items are different. In Λ (1) a neighbouring pair is similar, e.g., B ∼ C. In Λ (2) , B ∼ D and so forth. k can be regarded as the lag from B until the similar item. Fig 6 shows the serial position curves for lists Λ, Λ (1) , and Λ (2) . Note that the coding strength of B is increased by any other item which is similar to B, while the strengthening is greater, the smaller the lag is, i.e. the effect of C ∼ B on the coding strength of B is larger than the effect of D ∼ B.

Cued associative recall
In this task, the participant is presented a paired memory list A − X, B − Y, C − Z, . . . . After memorizing this list, a memory item, i.e. a member of some pair, is presented as a cue, and the participant is asked to identify the memory item, which was bound to that cue item. The memory state corresponding to the paired list is where a * x is the state representing the binding between items X and A in the list. Figure 7: Cued Recall: Given cues such as x, y, z, the corresponding activations α x * R , α y * R , α z * R are plotted, see eq 14. The cue x causes the activation α x * R to have a peak at the corresponding item, which is a. α R is the activation profile of the R-state.
When a memory item X is presented as a cue and the task is to retrieve the item which is bound to X in the list, consider the activation of The activation α x * R attains its maximal value for α x * R (a), see Fig. 7. Thus the cue X activates the A component most, so that the participant will answer " X is bound to A." , with some probability. Analogously, if the cue is Y , the activation α y * R attains it maximum in B, so that B is retrieved, and so forth. These maximal points form a curve, which is identical to the activation gradient α R .

Retrieval from similar contexts
Assume that the paired list A − X, B − Y, C −X, D − Z, . . . is given, in which items A and C are bound to similar contexts X andX. The corresponding state yields Cueing with X will not only retrieve A but also C, just to a lesser extend. The effect of cueing with x is displayed when considering the activation gradient α x * R , see Fig. 8. The gradient has two peaks, one at a and a weaker one at c, saying that cueing with X reveals two items, A and C.
Cueing with Y uncovers only one, which is B. The activation profile α R of the R state by distinct list items is shown as a reference.
In the recall task, the participant has to make a choice between the two alternative items bound to X. Thus invoking Luce's choice axiom, the probability to recall X yields which is less than the probability to recall a without an alternative. The existence of an item similar to the cue impairs the corresponding recall.

Putting things together: The serial position curve
During memorizing a list, the two states R and L are constructed. Since there is no a priori reason to favour one over the other, I assume that both cognitive states L and R coexist and are the components of a memory state M, A single cue thus activates both components. The total activation of the memory state M is a linear combination of the activation gradients of its two components.
where ρ and are non-negative parameters governing the mixture of respective activations. The response function to a cue is Φ(x|α M ), so that the serial position curve is the graph Φ(x|α M ), where x is a state representing a list item, see Fig. 9. Figure 9: The Serial Position Curve is simulated for different pairs of parameters according to eq. 18. It shows a strong recency effect and a weak primacy effect for ρ = 0.4, λ = 0.9, while for ρ = 0.9, λ = 0.4 there is a strong primacy effect and a weak recency effect.
The serial position curve thus results from the linear combination of the primacy gradient α R and the recency gradient α L . As seen in Fig 9, a large ρ together with a small makes the recency effect, while a small ρ together with a large leads to a prominent primacy effect. The relative strength of the primacy and the recency effect will generally depend on the experimental set-up, including the task to be performed. For example in recognition and in cued (probed) recall the serial position curve shows a strong recency effect, while the primacy effect is weak. Strong primacy effects are seen in forward recall, while recency effects are strong in backward or in free recall.

Conclusion and out-look
In the previous sections, an elementary algebra ( eq. 4 ) for storage and retrieval of information in basic cognitive tasks was proposed. The aim was not to present a full-blown model but to investigate how far one can get with the algebra alone.
Item information and associative information are represented by two operations, bundling and binding, respectively. If bundling is realised by an associative operation such as ordinary (vector-) addition, information about sequential order is lost. On the other hand, tasks such as serial recall require that order information. Consequently in corresponding models order information has be has to be implemented separately. This can be achieved by postulating serial position markers, chaining by associative mechanisms between consecutive items, or weight functions varying over serial position governing the recency and the primacy effect. This is different in the approach presented: Information about sequential ordering is preserved. This is due to the non-associativity of the addition operation by which item information is bundled into a memory state. Reading from that state thus reveals order information necessary to related tasks, which is represented by corresponding gradients. Activation gradients are implied rather than postulated separately. The serial position curve comes as a linear combination of both. Its shape is concave-up and asymmetric as observed as a typical experimental finding, see As already mentioned, the aim was not to present a full-blown model but to investigate how far one can get with the algebra alone. So it does not come as a surprise that several experimental observations were not captured. For example, while the recency effect does not depend on list length, the primacy effect does. This robust finding cannot be explained by our algebra alone but needs an additional assumption about attention, which then imposes an additional constraint on den attention gradient. Furthermore, serial recall can not be described by our algebra alone but needs an additional assumption such as output-suppression, as supposed in many models, or an other feedback mechanism, see [3].
The cognitive algebra proposed appears to provide a reasonable basis for modelling since it generically implies several features that fit empirical observations quite well, in a qualitative sense in that no attempt was made to fit data. Modelling then could consist in carefully adding assumptions on top of the cognitive algebra such as discussed above.