Entropy landscape of solutions in the binary perceptron problem
  The statistical picture of the solution space for a binary perceptron is
studied. The binary perceptron learns a random classification of input random
patterns by a set of binary synaptic weights. The learning of this network is
difficult especially when the pattern (constraint) density is close to the
capacity, which is supposed to be intimately related to the structure of the
solution space. The geometrical organization is elucidated by the entropy
landscape from a reference configuration and of solution-pairs separated by a
given Hamming distance in the solution space. We evaluate the entropy at the
annealed level as well as replica symmetric level and the mean field result is
confirmed by the numerical simulations on single instances using the proposed
message passing algorithms. From the first landscape (a random configuration as
a reference), we see clearly how the solution space shrinks as more constraints
are added. From the second landscape of solution-pairs, we deduce the
coexistence of clustering and freezing in the solution space.
