Asymptotic Learning Curve and Renormalizable Condition in Statistical
  Learning Theory
  Bayes statistics and statistical physics have the common mathematical
structure, where the log likelihood function corresponds to the random
Hamiltonian. Recently, it was discovered that the asymptotic learning curves in
Bayes estimation are subject to a universal law, even if the log likelihood
function can not be approximated by any quadratic form. However, it is left
unknown what mathematical property ensures such a universal law. In this paper,
we define a renormalizable condition of the statistical estimation problem, and
show that, under such a condition, the asymptotic learning curves are ensured
to be subject to the universal law, even if the true distribution is
unrealizable and singular for a statistical model. Also we study a
nonrenormalizable case, in which the learning curves have the different
asymptotic behaviors from the universal law.
