A unified theory of information transfer and causal relation
  Information transfer between coupled stochastic dynamics, measured by
transfer entropy and information flow, is suggested as a physical process
underlying the causal relation of systems. While information transfer analysis
has booming applications in both science and engineering fields, critical
mysteries about its foundations remain unsolved. Fundamental yet difficult
questions concern how information transfer and causal relation originate, what
they depend on, how they differ from each other, and if they are created by a
unified and general quantity. These questions essentially determine the
validity of causal relation measurement via information transfer. Here we
pursue to lay a complete theoretical basis of information transfer and causal
relation. Beyond the well-known relations between these concepts that
conditionally hold, we demonstrate that information transfer and causal
relation universally originate from specific information synergy and redundancy
phenomena characterized by high-order mutual information. More importantly, our
theory analytically explains the mechanisms for information transfer and causal
relation to originate, vanish, and differ from each other. Moreover, our theory
naturally defines the effect sizes of information transfer and causal relation
based on high-dimensional coupling events. These results may provide a unified
view of information, synergy, and causal relation to bridge Pearl's causal
inference theory in computer science and information transfer analysis in
physics.
