{'fraction_non_alphanumeric': 0.027140001211607197, 'fraction_numerical': 0.008178348579390562, 'mean_word_length': 4.442631058358061, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Causal Future Prediction in a Minkowski Space-Time

Estimating future events is a difficult task. Unlike humans, machine learning approaches are not regularized by a natural understanding of physics. In the wild, a plausible succession of events is governed by the rules of causality, which cannot easily be derived from a finite training set. In this paper we propose a novel theoretical framework to perform causal future prediction by embedding spatiotemporal information on a Minkowski space-time. We utilize the concept of a light cone from special relativity to restrict and traverse the latent space of an arbitrary model. We demonstrate successful applications in causal image synthesis and future video frame prediction on a dataset of images. Our framework is architecture- and task-independent and comes with strong theoretical guarantees of causal capabilities.


Introduction
In many everyday scenarios we make causal predictions to assess how situations might evolve based on our observations and experiences. Machine learning has not been developed to this level yet, though, automated, causally plausible predictions are highly desired for critical applications like medical treatment planning, autonomous vehicles and security. Recent works have contributed machine learning algorithms for the prediction of the future in sequences and for causal inference [6]. One major assumption that many approaches implicitly adopt, is that the space of the model representation is a flat Euclidean space of N dimensions. However, as shown by Arvanitidis et al. [1], the Euclidean assumption leads to false conclusions as a model's latent space can be better characterized as a high dimensional curved Riemannian manifold rather than an Euclidean space. Furthermore, the Alexandrov-Zeeman theorem [24; 5] suggests that causality requires a Lorentzian group space and advocates the unsuitability of Euclidean spaces for causal analysis.
In this paper, we present a novel framework that changes the way we treat hard computer vision problems like the continuation of frame sequences. We embed information on a spatio-temporal, high dimensional pseudo-Riemannian manifold -the Minkowski space-time -and utilize the special relativity concept of light cones to perform causal inference. We focus on temporal sequences and image synthesis to exhibit the full capabilities of our framework.
• We extend representation learning to spatio-temporal Riemannian manifolds that follow the ideas of the Minkowski space-time while being agnostic towards the used embedding architecture and the prescribed task.
• We introduce a novel utilization of the concept of light cones and use them for convincing frame synthesis and plausible prediction of future frames in video sequences.
• We provide theoretical guarantees about the causal properties of our model and demonstrate a causal inference framework.

Related Works
High dimensional Riemannian manifolds for machine learning are utilized by a few major works. Arvanitidis et al. [1] show evidence that more general Riemannian manifolds characterize learned latent spaces better than an Euclidean space. Their work however, utilizes generators that have been trained under an Euclidean assumption. Contrary to that, Nickel et al. [10] introduce the use of a Poincaré ball for hierarchical representation learning on word embeddings, showing superior performance in representation capacity and generalization ability while employing a Riemannian optimization process. In [11], Nickel et al. extend the previous work to a Lorentzian manifold as this offers improvements in efficiency and stability of the distance function. In this paper we accept the argument made by Nickel et al. but extend it as we argue in Section 3.3 that causal inference requires a Lorentzian group space [24].
Ganea et al. [3] embed word information on a Poincaré ball and form entailment cones. The authors propose to work with Directed Acyclical Graphs (DAG) and strive for non overlapping cones in a Poincaré ball. In contrast to this, we encourage overlapping light cones in a Lorentzian manifold to model future events.
Sun et al. [18] use a space-time idea similar to ours but interpret the time axis as a ranking rather than as temporal information. Their method is intended for dimensionality reduction and does not generate further samples, or considers causal relationships between sampling points. Finally, Mathieu et al. [8] train a Variational Autoencoder (VAE) constrained to a Poincaré ball while also employing the appropriate Riemannian equivalent to a normal distribution as well as Riemannian optimization. We consider this work as the closest related since it is the only approach that has shown good performance in the image domain.
In the Computer Vision focused field of future frame prediction for video sequences, [6] propose the causal InfoGAN which, however, lacks theoretical guarantees of causal abilities. [4] aims at predicting the probabilistic bottlenecks where the possible futures are constrained instead of generating a single future. Similarly, we are not attempting to predict a single future, rather we predict all plausible futures in a way that naturally enables us to identify all probabilistic bottlenecks; see Section 3.6. In other works concerned with video continuation, [9; 22] use CNNs to regress future frames directly, while [20] introduce an LSTM utilizing the difference ∆ between frames to predict motion. Further works include the use of optical flow [7] or human pose priors [21]. The autoregressive nature of these methods results in accumulated prediction errors that are propagated through the frames the further a sequence is extended. Beyond a few frames, these approaches quickly lose frame-to-frame consistency. In order to mitigate these limitations, works like [23] propose generative models to predict future frames and [19] offers a generative model that disentangles motion and content. Neither can infer the causal implications of their starting positions.

Background
For an in-depth review of manifolds we invite the reader to refer to S. Carroll's textbook [2]; in this section we will briefly provide some key concepts of the used differential geometry.
Manifold: A manifold M of dimensions n is a generalization of the concept of the surface in a non-Euclidean space and is characterised by a curvature c. The manifold group that this paper considers is constantly flat.
Tangent Space: The tangent space T x M is a vector space that approximates the manifold M at a first order.
Riemannian Metric: A Riemannian metric g is a collection of inner products T x M × T x M → R. It can be used to define a global distance function as the greatest local bound of the length l of all the smooth curves γ connecting points x, y ∈ M . Note that the length is defined as l(γ) = 1 0 g γ(t) (γ (t), γ (t)) dt. Geodesic: Geodesics are the generalizations of straight lines in Euclidean space and define the shortest path between two points of the manifold. They can also be defined as curves of constant speed.
Exponential map: The exponential map exp x :

Causal Inference
Causal inference refers to the investigation of causal relations between data. There is a rich literature on machine learning and causal inference ranging from association of events to counterfactuals [14; 12]. Briefly we observe two equivalent approaches towards causality in machine learning: Structural Causal Models [12; 13] which rely on Directed Acyclical Graphs (DAG) and Rubin Causal Models [15] which rely upon the potential outcomes framework. In this paper we will be focusing on the latter. In the potential outcomes framework as established by [15] multiple outcomes Y of X are contrasted in order to deduce causal relations between Y and X . As we will show, our proposed method provides the theoretically guaranteed infrastructure to create a Rubin Causal Model. In addition, as our method is able to operate in a future as well as a past regime it enables the formation of counterfactual questions, i.e., what would Y be if X had happened instead.

On the choice of space
In his seminal 1964 work, E.C. Zeeman [24] makes the case that the causality group RM that arises from the concept of partial ordering in a Minkowski space-time implies an inhomogenous Lorentz group as the symmetry group of RM . We highlight the explicit mention of Zeeman on the unsuitability of an Euclidean topology to describe RM due to its local homogeneity, which does not arise in RM . In [5] the authors prove that from observable causality we can reconstruct the Minkowski space-time. Hence, we are in a position to argue that the use of a Minkowski spacetime for embeddings, which belongs to the inhomogenous Lorentz group, would reinforce causal inference capabilities.
We define our Minkowski space-time to be characterized by the metric of Eq. 1 with the element −1 denoting the temporal dimension and +1 elements the spatial dimensions. We extend [11] and argue that the use of the Lorentzian manifold, which coincides with the Minkowski space-time, is both more efficient as an embedding as well as enabling causal arguments,

Minkowski Space-Time and Causality
Mathematically a space can be described by its metric, which defines the way the inner product of two vectors in this space is determined, i.e. the way we calculate distances. Consequently, the inner product ., . η of two vectors a and b in 1 + 3D Minkowski space-time can be defined as where the coordinate 0 is understood to be the time coordinate.
One of the consequences of endowing the latent space with a Minkowski-like metric is the emergence of causality in the system. This property can be more readily seen by employing the concept of proper time. Given a manifold M endowed with a Minkowski metric η µν , we define the proper time τ . This is the time measured by an observer following along a continuous and differentiable path C(s) parametrized by s ∈ [0, 1] between two events {x, y} ∈ M such that C(0) = x, C(1) = y, In order to ensure τ ∈ R, we require i dx 2 i ≤ dx 2 0 , where i ∈ 1, 2, ..., d. Therefore, the rate of change |dx|/dτ in the spatial coordinates is capped by the time evolution of the system. In other words, there exists a maximum speed limit which C must obey at every point. Further, it means that there exist pairs of space-time points x, y which cannot be possibly connected by a valid path C , lest τ / ∈ R. In order to describe this phenomenon we borrow the concept of a light cone from special relativity. The set of solution paths {C 0 (s)} such that C 0 (0) = (t 0 , x 0 ) and τ C0 = 0 describe the fastest any particle or piece of information can travel within the system starting from (t 0 , x 0 ). This boundary is known as the light cone, and is such that ∂R = {C 0 (s)}, where R is the causal region of the point (t 0 , x 0 ). Every space-time point x ∈ R is said to be within the light cone. As shown by (3), no valid path C(s) can cross ∂R. Thus, two space-time points can only influence each other if they lie within each other's light cone, that is, if they can be connected by a valid path C. The region R splits into two disjoint sets: R + and R − . R + lies within the future light cone of a particle at time t 0 , and thus includes all of the points (t 1 , x 1 ) ∈ R such that t 1 > t 0 . Conversely, R − includes the points (t 2 , x 2 ) ∈ R such that t 2 < t 0 and characterizes the past light cone of a particle at time t 0 .
If we have two space-time vectors x = (t 0 , x 0 ) and y = (t 1 , x 1 ) we can describe their relation as timelike when x, y < 0, spacelike when x, y > 0 and lightlike when x, y = 0. A timelike position vector lies within the light cone of a particle at the origin of the system. A spacelike vector lies outside of it, and a lightlike vector lies exactly at its edge. One can then generalize this idea beyond the origin, and thus compute the inner product of the difference between two space-time vectors x − y ≡ (∆t, ∆r) , i.e., y − x, y − x = −∆t 2 + |∆r| 2 . Hence, when the separation of the vectors x and y is timelike, they lie within each other's causal region. In that case we can argue that there is a path for particle x, that belongs in the model that defines the latent world of represented data, to evolve into particle y within a time period ∆t. Thus, by constructing the light cone of an initial point x we can constrain the space where the causally resulting points may lie. We can then see that this mathematical construction of the latent space naturally enforces that the velocity of information propagation in the system be finite, and that a particle can only be influenced by events within its past light cone, i.e. the model is causal. By mapping this into a machine learning perspective we argue that in a latent space that is built to follow the Minkowski space-time metric an encoded point can then be used to create a light cone that constrains where all the causally entailed points may be encoded to or sampled from.

On Intersecting Cones
A light cone can be constructed with each point of the latent space as its origin. Consider point x 0 to be an initial point derived from, for example, an encoded frame f 0 from a video sequence: by constructing the light cone C 0 around x 0 we are able to deduce where the various causally related x 0+t points might lie. By setting t to be a specific time instance, we are able to further constrain the sub-space to points that lie inside of the conic section. They are causally plausible results of point x 0 within the time t. Geometrically, we can visualize this as a plane cutting a cone at a set time. We visualize this in Figure 1a.
A second point x 1 that lies inside the light cone of x 0 can be derived from an encoded frame f 1 . Similar to x 0 we construct the light cone C 1 whose origin is x 1 . We then define the conic intersection CS = C 0 ∩ C 1 . Following the causality argument, we deduce that the enclosed points in CS are causally related to both x 0 and x 1 as they lie in the light cones of both. In addition, by constraining the intersecting time plane, we constrain the horizon of future prediction.  Consequently, we propose Algorithm 1 as a method of future frame prediction using light cones on a Minkowski space-time latent space. We graphically represent Algorithm 1 in Figure 1b.

On the Entropy and the Aperture of Cones
When considering the intersection of the cones in Algorithm 1 it is vital to examine the aperture of the cone at time T . For simplicity, we assume that the gradient of the side of the cone is 45 • for all cones. However, such an assumption implies that each frame and hence each cone evolves with the same speed and can reach the same number of states at a given time. For real world scenarios this is not necessarily true as, for example, the possible states in t + 1 for a ball rolling constraint by rails are less than a ball rolling on a randomly moving surface. Hence, the actual gradient of the cone depends on the number of states that are reachable from the state depicted in frame t. This quantity is also known as the thermodynamic entropy of the system. It is defined as the sum of the states the system can evolve to. Calculating the thermodynamic entropy of a macro-world system as in a real world dataset is not trivial and we are not aware of any appropriate method to compute this at the time of writing. Hence, we are forced to make the aforementioned assumption of 45 • .
However, given a frame sequence F , a set of counter example frames CF and following Algorithm 1 but omitting the sampling steps, it is possible to build more accurate light cones in a contrastive manner. Hence, it is possible to acquire a proxy for the thermodynamic entropy of the system. We note that the proxy can only be accurate to a certain degree as any frame sequence is not able to contain enough information to characterize the full state of the world.

Training
Our proposed algorithm is invariant to the method used to train the embedding. In an ideal scenario, we require an encoder-decoder pair that is able to map any image to a latent space and to reconstruct any latent code. For the purposes of this paper's evaluation we have chosen the method by Mathieu et al. [8] as our baseline embedding, as it is the only approach that has shown good image domain performance.
Mathieu et al. [8] construct a Variational Auto Encoder (VAE) that enforces the latent space to be a Poincaré Ball. We analyze the properties of the Poincaré ball in the supplementary material. It can be shown [11] that a n−dimensional Poincaré ball embedding can be mapped into a subspace of the Minkowski space-time by an orthochronous diffeomorphism m : P n → M n , and back with the inverse m −1 : where x i is the i-th component of the embedding vector.
We extend [8] to enforce the embedding to a subspace of the Minkowski space-time by utilizing Eq. 4 and 5. We treat the space's dimensionality as hyper-parameter and tune it experimentally. We establish that the optimal embedding of our data can be achieved in an 1 + 8 dimensional space i.e. 1 time and 8 space dimensions. The model consists of a MLP with a single hidden layer and was trained with the Riemannian equivalent of the Adam optimizer [18] with a learning rate of 5e − 4.
Training the model with Moving MNIST requires on a Titan RTX Nvidia GPU less than 1 hour.

Inference
Our proposed Algorithm 1 is executed during inference as it does not require any learned parameters. We sample from a Gaussian distribution wrapped to be consistent with our Minkowski space-time in a manner similar to [8], details can be found in the supplement. Inference can be performed in about 0.5 s per intersecting cone in an exponential manner.

Dataset
As a proof of concept we use a custom version of the Moving MNIST dataset [17]. Specifically we employ 10.000 sequences consisting of 30 frames each, making a total of 300.000 frames. Each sequence contains a single digit. The first frame is derived from the training set of the original MNIST dataset, while the subsequent frames are random continuous translations of the digit. Construction of the test set followed the same procedure with the first frame derived from the test set of the original MNIST dataset. We created 10.000 testing sequences of 30 frames each. Each frame is 32 × 32 while the containing digits range from 18px − 25px We further use the KTH action recognition dataset [16] to highlight the real world capabilities of our method. We focus on the walking and handwaving actions and use all 4 distinct directions. Different person identities are used in the train-test split.

Experiment 1: Single Cone Image Synthesis
In the first experiment we evaluate the ability of the light cone to constrain the latent space such that samples lying inside the cone are reasonably similar to the original frame. We train our model with 1+8 latent dimensions. Following standard VAE sampling we produce 100.000 random samples using a wrapped normal distribution. As expected, the tighter the imposed time bound is, the fewer samples are accepted. We note that for t = 2 only 2 samples were accepted, for t = 10 our method accepts N = 31% of the samples and for t = 20, N = 71%. In Figure 2a we exhibit qualitative results for Experiment 1. We note that as the time limit increases we observe higher variability, both in terms of morphology and location of the digits, while the identity of the digit remains the same. This is in accordance with the theory that the "system" would

Experiment 2: Intersecting Cones
In the second experiment we evaluate the ability of our algorithm to predict frames by intersecting light cones. There is no unique path a system might evolve in time. Our algorithm does not aim at producing a single future, rather it is able to produce multiple plausible futures. At a single time instant we can find any number of probable frames that extend a sequence. Hence, the choose step of Algorithm 1 depends on the target application. In this experiment to guide the choice of frames we map the sampled points to image space and compare the structural similarity of them with the original frame of t = 0. We adopt a simple manner to choose the next step and we do not provide the model with any further conditioning information to highlight the default strengths of the proposed algorithm. In an online inference scenario the reference frame could be updated as ground truth frames become available.
In Figures 2b and 3a we exhibit qualitative results of our algorithm when intersecting 2 and 5 cones respectively. In Figure 2b each set of results evaluates a specific movement, vertical or horizontal.
In Figure 3a we exhibit the case of intersecting 5 cones. As this scenario allows up to 10 time steps for our model to evolve we notice a great number and more varied results. In the first two rows the depicted digits bounce while moving towards one direction. In the third row the digit 0 exhibits morphological changes and in the fourth row the digit 6 gradually moves its closing intersection upwards to become a 0. As our model is only trained with single frames of MNIST digits it is not constrained to show only movement or morphological changes. Rather it can vary both as seen in Figure 3a. The transmutation of the digit 6 to 0 is a probable, albeit unwanted, outcome under certain scenarios. In addition, we note that we are not providing any labels or additional information to the model during inference. In principle, one could condition the model to produce probable future frames by tuning the choose procedure of Algorithm 1.

Experiment 3: Realistic video data
As a final experimentation we use the KTH action dataset. Examples of the performance of the proposed algorithm are shown in Fig. 3b. Due to the computational constraints of the Poincaré VAE, which we are using as a base model, we are limited to one action at a time during training. We note how our algorithm retains characteristics like the shade of gray of the clothing while producing plausible futures. Each frame differs to the previous by 2 time instances giving ample time for the subject to change directions. We believe that with a higher capacity network a similar performance can be achieved on more complex scenes and higher resolution videos.
(a) Moving MNIST (b) KTH movement video sequences dataset Figure 3: Samples from Experiment 2 (a) and Experiment 3 (b). We are intersecting 5 cones trained on the moving MNIST dataset in (a) and the KTH movement video sequence dataset in (b). The latter is representative for a real-world use-case scenario. Next to each row we added an explanatory caption about the type of observed movement. Differences in image brightness in (b) are due to PyTorch's contrast normalization in the plotting function.

Discussion
As the model is only trained as a VAE on single frames and not on sequences, the notion of time is not encoded in the weights of the network. Hence, all the resulting movement and predictive abilities are derived from our proposed algorithm and the natural embedding abilities of the Minkowski spacetime.
We emphasize the time-agnostic nature of our algorithm. Our predictions are constrained in time but are probabilistic in nature. The proposed algorithm is able to produce multiple plausible futures. We believe this is a very important feature for future prediction and sequence extrapolation techniques as it can be used as an anomaly detection technique. Specifically, if one of the produced futures includes a hazardous situation, an automated system can adapt in order to avoid an outcome, enabling for example defensive driving capabilities in autonomous vehicles.
Even though our method is in principle auto-regressive, it does not suffer from the accumulation of errors as it is both probabilistic and relies on efficient latent space sampling rather than the ability of a neural network to remember structural and temporal information about the image.
Furthermore, we believe that the quality of the predicted frames as well as the definition of the subspace from which the samples should be derived could be improved by incorporating the inferred thermodynamic entropy of the frame. We will explore the link between the information and thermodynamic entropy in future work. In addition, even though our framework is architecture agnostic, a customized architecture for the prediction task would be an intriguing direction.
Finally, as our model allows us to find all probable scenarios that might exist, it can be used as a causal inference tool in the "potential outcomes" framework [15]. Given a state we are able to probe possible scenarios and investigate plausible outcomes, hence, deduce causal relations within the data. In addition, by using the past light cone R − , we are able to probe the events that could have led to an observed state enabling counterfactual analysis.

Conclusion
Machine learning techniques are able to build powerful representations of large quantities of data. We leverage this ability and propose that hard computer vision problems can be approached with minimal learning in an architecture agnostic manner. Strong mathematical and physical priors are the key. In this paper, we extend early Riemannian representation learning methods with the notion of Minkowski space-time as it is more suitable for causal inference. We further propose a novel algorithm to perform causally plausible image synthesis and future video frame prediction utilizing the special relativity concept of light cones. We showed our algorithms' capabilities both in the synthetic Moving MNIST dataset as well as the real-world KTH dataset.

Broader Impact
In this paper we address an important theoretical topic. We believe that this work by itself does not have any ethical implications. We are proposing a generalized framework that can be used during causal reasoning applications and for future frame prediction. It is our conviction that applications using our framework that automate any use of equipment or provide medical diagnosis should have proper safety procedures in place to ensure the well-being of humans and animals. Our work incorporates insights from physics into the field of machine learning, allowing the development of a novel view on representation learning. This way, algorithms may gain insights about the world that are usually not consciously tractable by humans. There is no conflict of financial interests stemming from this work.

Step-by-Step Visualization of the Proposed Algorithm
In order help the reader further understand the intuition behind our proposed algorithm we will be conducting a mental experiment with the help of some visual queues. Note the figures will follow the convention of 2+1 dimensional euclidean space, simply for ease of understanding.

Poincaré Ball
As stated in the main paper Riemannian Geometry can be seen as a curved generalization of Euclidean space. In this section we will be focusing on the Poincaré ball Riemannian manifold, as it forms the basis of our implementation. We note that there is no theoretical reason why we extend the Poincaré VAE [? ] other than simplicity of implementation and proven results in the image domain.
Many works rely on a Poincaré ball, as has been argued [? ? ? ] that embedding the latent space on a Poincaré Ball, that is, a hyperbolic space with negative curvature, allows one to naturally embed continuous hierarchical relationships between data points. This follows from the qualitative properties of such a hyperbolic space: 1. The entirety of the Poincaré Ball B d c is contained within a hypersphere of radius 1/ √ c and dimensionality d, in what amounts to compactification of infinite space. 2. The distance function (and thus area element) of this space grows rapidly as one approaches the edges of B d c , such that reaching the edge would require traversing an infinite distance in latent space. 3. This behaviour naturally emulates the properties of hierarchical trees, whose size grows exponentially as new branches "grow" from previously existing branches.
Quantitatively, the space B d c is endowed with a metric tensor g c which relates to flat Euclidean space as follows Figure 4: Embed a third frame F 2 and increase the circles of frames 0,1 the fourth frame has to lie in the intersection of the circles. where r is a d-dimensional vector in latent space and g e (r the Euclidean metric. As a result, the distance element in B d c may be written, in spherical coordinates, where r = |r| is the radius from the origin of the space and dΩ d is the differential solid angle element in d dimensions. As explained qualitatively above, it easy to see that the distance element diverges as r → 1/ √ c, thus encoding the infinite hypervolume contained near the edges of the Poincaré Ball. Furthermore, as c → 0, the radius of the Poincaré Ball becomes infinity and g c (z) → g e (z), up to a constant rescaling of the coordinates. Let γ : t → γ(t) be a curve in B d c , where t ∈ [0, 1] such that its length is defined by Figure 6: Two potential causal paths from points a,b to a new point c. Note that if its these points represent a sequence then the causal path will have to pass from A → B → C whereĝ c is the matrix form of g c and v ≡ dr dt is the trajectory's velocity vector. In component form, this reads: where x µ represents each coordinate, g c µν is the component form ofĝ c and in the last step we have used (1). In hyperbolic space, "straight lines" are defined by geodesics γ g (t), that is, curves of constant speed and least distance between points x and y. Thus, γ g (t) = argmin [L(γ(t))] γ(1)=y γ(0)=x and dγ(t) dt = 1.
With (4) and (5), one may show that the distance function d c (x, y) between two points x and y on B d c can be computed to yield:

Wrapped Normal
While embedding data on a Riemannian space with the use of a Riemannian VAE, it is important to embed the used distribution in the space as well. Multiple ways have been proposed to perform this process. We will be following the wrapped normal distribution [? ? ].
In this approach a normal distribution is mapped onto the manifold using the manifold's exponential map. Given a normal distribution z e ∼ N (0, Σ) then the Riemannian sample z = exp c µ ( ze λ c µ ). The distribution's density can be described as: ) d−1 As c → 0 the Euclidean normal distribution can be obtained.

Architectural Considerations
In order to properly embed information on a manifold a set of considerations have to be taken into account as developed by Ganea et al. [? ]. In this paper we are following the architectural guidance of [? ? ] about the last layer of the encoder and the first of the decoder. Specifically in the encoder we use the frechet mean as calculated by the exponential mapping exp c 0 and a solftplus variance σ. In terms of the decoder we utilize the gyroplane layer as developed by [? ? ]. Our architecture follows the consideration from [? ] with the additions of the extended mapping using equations 5,6 from the main paper and increased capacity of the hidden layers as our input is 32 × 32 rather than the original 28 × 28.

Optimization:
In terms of optimization we tested both a Riemannian stochastic gradient descent as seen in [? ] and [? ]'s approach of using the exponential mapping to bring the model's parameters onto the manifold. As there was inconsequential practical differences between the two approaches and as both are theoretically sound we opted with [? ]'s approach for computational simplicity.

Online learning and Anomaly Detection
A limitation of causal analysis is that it fails to include unseen causal sources. For example in the case of an autonomous vehicle simulation the addition of a second car by the researcher would entail a causal anomaly in the world model of the autonomous vehicle where it was the only vehicle in existence. It is obvious that causal future predictions like these are impossible to perform as the system is assumed to be impervious to modifications from outside sources.
However, by presenting an anomaly like that and mapping it on our world model we are able to raise an anomaly flag. If an observation is made that falls outside the perceived light cone of the system then if an event like this has happened before we are able to adapt the aperture of the cone, or as in the case of the scientist inserting a new vehicle, re-structure the world model of the system based on the new observation. This can be considered as a method of online fine-tuning. We believe that beyond fully retraining our model we are able to adjust the embedding space and subsequently the light cones by modifying the metric with the use of free parameters. Investigations on which method is optimal will follow in future work.

Code
The proposed algorithm was implemented in PyTorch and the code will be made available by the time of the conference.