Empirical Bayes Method for Boltzmann Machines
  In this study, we consider an empirical Bayes method for Boltzmann machines
and propose an algorithm for it. The empirical Bayes method allows estimation
of the values of the hyperparameters of the Boltzmann machine by maximizing a
specific likelihood function referred to as the empirical Bayes likelihood
function in this study. However, the maximization is computationally hard
because the empirical Bayes likelihood function involves intractable
integrations of the partition function. The proposed algorithm avoids this
computational problem by using the replica method and the Plefka expansion. Our
method does not require any iterative procedures and is quite simple and fast,
though it introduces a bias to the estimate, which exhibits an unnatural
behavior with respect to the size of the dataset. This peculiar behavior is
supposed to be due to the approximate treatment by the Plefka expansion. A
possible extension to overcome this behavior is also discussed.
