Non-trivial informational closure of a Bayesian hyperparameter
  We investigate the non-trivial informational closure (NTIC) of a Bayesian
hyperparameter inferring the underlying distribution of an identically and
independently distributed finite random variable. For this we embed both the
Bayesian hyper-parameter updating process and the random data process into a
Markov chain. The original publication by Bertschinger et al. (2006) mentioned
that NTIC may be able to capture an abstract notion of modeling that is
agnostic to the specific internal structure of and existence of explicit
representations within the modeling process. The Bayesian hyperparameter is of
interest since it has a well defined interpretation as a model of the data
process and at the same time its dynamics can be specified without reference to
this interpretation. On the one hand we show explicitly that the NTIC of the
hyperparameter increases indefinitely over time. On the other hand we attempt
to establish a connection between a quantity that is a feature of the
interpretation of the hyperparameter as a model, namely the information gain,
and the one-step pointwise NTIC which is a quantity that does not depend on
this interpretation. We find that in general we cannot use the one-step
pointwise NTIC as an indicator for information gain. We hope this exploratory
work can lead to further rigorous studies of the relation between NTIC and
modeling.
