Discovering Causal Structure with Reproducing-Kernel Hilbert Space
  $\epsilon$-Machines
  We merge computational mechanics' definition of causal states
(predictively-equivalent histories) with reproducing-kernel Hilbert space
(RKHS) representation inference. The result is a widely-applicable method that
infers causal structure directly from observations of a system's behaviors
whether they are over discrete or continuous events or time. A structural
representation -- a finite- or infinite-state kernel $\epsilon$-machine -- is
extracted by a reduced-dimension transform that gives an efficient
representation of causal states and their topology. In this way, the system
dynamics are represented by a stochastic (ordinary or partial) differential
equation that acts on causal states. We introduce an algorithm to estimate the
associated evolution operator. Paralleling the Fokker-Plank equation, it
efficiently evolves causal-state distributions and makes predictions in the
original data space via an RKHS functional mapping. We demonstrate these
techniques, together with their predictive abilities, on discrete-time,
discrete-value infinite Markov-order processes generated by finite-state hidden
Markov models with (i) finite or (ii) uncountably-infinite causal states and
(iii) continuous-time, continuous-value processes generated by thermally-driven
chaotic flows. The method robustly estimates causal structure in the presence
of varying external and measurement noise levels and for very high dimensional
data.
