I believe that the derivative of a Gaussian process (GP) is a another GP, and so I would like to know if there are closed form equations for the prediction equations of the derivative of a GP? In particular, I am using the squared exponential (also called the Gaussian) covariance kernel and want to know about making predictions about the derivative of the Gaussian process.

  • $\begingroup$ What do you mean by the derivative of the GP? do you randomly generate a curve from the BP, $x(t)$, and then take the derivative? $\endgroup$ – Placidia Nov 9 '15 at 3:00
  • $\begingroup$ @Placidia, no I mean calculating $\frac{\partial x(t)}{\partial t}$, which I believe should be another Gaussian process $\endgroup$ – user30490 Nov 9 '15 at 5:23
  • $\begingroup$ Good question. However I seem to recall that Brownian motion is both a GP and nowhere differentiable. So I'm not sure that there could be a generic expression. Of course x(t)-x(t-h) should be a Gaussian so it should be possible given the covariance function to think of probabilities about it for a given h. $\endgroup$ – conjectures Nov 9 '15 at 16:19
  • $\begingroup$ @conjectures, that's why I specifically said I have a GP where the kernel function is the squared exponential (since I know that that one is infinitely differentiable) and was really only looking for the derivative case in my example. But good point none the less! $\endgroup$ – user30490 Nov 9 '15 at 16:55

The short answer: Yes, if your Gaussian Process (GP) is differentiable, its derivative is again a GP. It can be handled like any other GP and you can calculate predictive distributions.

But since a GP $G$ and its derivative $G'$ are closely related you can infer properties of either one from the other.

  1. Existence of $G'$

A zero-mean GP with covariance function $K$ is differentiable (in mean square) if $K'(x_1, x_2)=\frac{\partial^2 K}{\partial x_1 \partial x_2}(x_1,x_2)$ exists. In that case the covariance function of $G'$ is equal to $K'$. If the process is not zero-mean, then the mean function needs to be differentiable as well. In that case the mean function of $G'$ is the derivative of the mean function of $G$.

(For more details check for example Appendix 10A of A. Papoulis "Probability, random variables and stochastic processes")

Since the Gaussian Exponential Kernel is differentiable of any order, this is no problem for you.

  1. Predictive distribution for $G'$

This is straightforward if you just want to condition on observations of $G'$: If you can calculate the respective derivatives you know mean and covariance function so that you can do inference with it in the same way as you would do it with any other GP.

But you can also derive a predictive distributions for $G'$ based on observations of $G$. You do this by calculating the posterior of $G$ given your observations in the standard way and then applying 1. to the covariance and mean function of the posterior process.

This works in the same manner the other way around, i.e. you condition on observations of $G'$ to infer a posterior of $G$. In that case the covariance function of $G$ is given by integrals of $K'$ and might be hard to calculate but the logic is really the same.

  • $\begingroup$ I do not understand your question. There is an explicit formula for the covariance function and the mean function given above (and in 9.4 of Rasmussen/Williams). As this is all there is to know and use a GP what else could you ask for? $\endgroup$ – g g Mar 24 '19 at 11:45
  • $\begingroup$ A process with this covariance is not differentiable. As stated in Section 1. of the answer the kernel function must be differentiable wrt both entries. The delta function is neither differentiable nor continuous. So $G'$ does not even exist. $\endgroup$ – g g Mar 26 '19 at 9:31
  • $\begingroup$ Is it possible that you confuse the mean function and the paths of the process? Note that the mean function is smoother than the paths and may be differentiable even though the process is not. But the mean function is a deterministic function, not a process, so there is no variance which can be calculated. $\endgroup$ – g g Mar 26 '19 at 9:37

It is. See Rasmussen and Williams section 9.4. Also, some authors argue strongly against the square exponential kenrnel - it is too smooth.

  • 1
    $\begingroup$ So is there a predictive distribution for the derivative? $\endgroup$ – user30490 Nov 9 '15 at 5:24

Your Answer
