# On the center of mass of Ising vectors

## Abstract

We show that the center of mass of Ising vectors that obey some simple constraints, is again an Ising vector.

PACS. 02.50$``$r Probability theory, stochastic processes, and statistics

PACS. 87.10$`+`$e General theory and mathematical aspects

PACS. 64.60Cn Order disorder transformations; statistical mechanics of model systems

Many problems in statistical mechanics are formulated in terms of an N-dimensional vector $`𝐉`$, with components $`J_i,i=1,\mathrm{},N`$ that take only binary values $`J_i=\pm 1`$. We will call such a vector an Ising vector. Its components represent for example a spin state (Ising model ), the occupancy of a site etc. In the thermodynamic limit $`N\mathrm{}`$, only a subset of all the possible configurations $`\{𝐉\}`$ are typically realized. In many cases, they are characterized by simple contraints of the form :

$$\underset{N\mathrm{}}{lim}\frac{𝐉𝐁}{N}=R\underset{N\mathrm{}}{lim}\frac{𝐉𝐉^{}}{N}=q,$$
(1)   
where $`𝐉`$ and $`𝐉^{}`$ are typical members of the subset, $`𝐁`$ is a symmetry breaking direction (imposed from the outside or arising through a phase transition), while $`q`$ and $`R`$ are physical properties describing the resulting macroscopic state (for example the magnetization or the density). In this letter, we focus on the center of mass of the vectors $`𝐉`$, that satisfy the above constraints. We report the surprising finding that it is an Ising vector whenever $`𝐁`$ is Ising.

To construct the center of mass we follow a Monte Carlo approach by choosing at random $`n`$ vectors $`𝐉^a,a=1,\mathrm{},n`$, that satisfy the constraints, and considering their center of mass:

$$𝐘=C^1\underset{a}{}𝐉^a,$$
(2)   
with the proportionality factor $`C=\sqrt{n+n(n1)q}`$, so that the normalization condition $`𝐘^2=N`$ is obeyed. In general the vector $`𝐘`$ is not Ising, but our contention is that it becomes Ising in the limit $`n\mathrm{}`$, provided $`𝐁`$ is Ising. In view of the permutation symmetry between the coordinate axes, it will be sufficient to prove that $`B_1Y_1=C^1_aB_1J_1^a`$ only takes the values $`+1`$ and $`1`$ in this limit. To show that this is the case, we focus our attention on the probability density $`P(y)`$ of the variable $`y=n^1_aB_1J_1^a`$, which differs by a factor $`n^1\sqrt{n+n(n1)q}\stackrel{n\mathrm{}}{}\sqrt{q}`$ from $`B_1Y_1`$. It is given by:

$`P(y)`$ $``$ $`{\displaystyle \left[\underset{a}{\overset{n}{}}d𝐉^aP_b(𝐉^a)\delta (𝐉^a𝐁NR)\right]}`$ (3)   
$`\times `$ $`\left[{\displaystyle \underset{a<b}{}}\delta (𝐉^a𝐉^bNq)\right]\delta \left(yn^1{\displaystyle \underset{a}{}}B_1J_1^a\right),`$   
where $`P_b`$ is the measure restricting to vectors with binary components,

$$P_b(𝐉)=\underset{j=1}{\overset{N}{}}\left[\frac{1}{2}\delta (J_j1)+\frac{1}{2}\delta (J_j+1)\right],$$
(4)   
and the proportionality constant has to be determined from the normalization condition $`_{\mathrm{}}^{\mathrm{}}𝑑yP(y)=1`$. The r.h.s. of (3) resembles an ordinary replica calculation , but with as limit of interest the number of replicas $`n`$ tending to infinity.

Rather than following the standard but lengthy calculations that are usual in this case, we present a more elegant, direct and expedient procedure. Since $`y=n^1_ax_a`$, with $`x_a=J_1^aB_1`$, we evaluate the joint probability density $`P(𝐱)`$ of the $`n`$-dimensional vector with binary components $`x_a`$, $`a=1,\mathrm{},n`$. Since all choices of the vectors $`𝐉^a`$ that satisfy the constraints can be realized, the Shannon entropy is maximized under the constraints (1). Hence $`P(𝐱)`$ is found by maximizing  its Shannon entropy $`_𝐱P(𝐱)\mathrm{ln}P(𝐱)`$, subject to the constraints:

$`x_a`$ $`=`$ $`{\displaystyle \frac{𝐉^a𝐁}{N}}=R`$   
$`x_ax_b`$ $`=`$ $`{\displaystyle \frac{𝐉^a𝐉^b}{N}}=q(a<b).`$ (5)   
One finds:

$$P(𝐱)=Z^1\mathrm{exp}\left[\underset{a}{}\widehat{R}_ax_a+\underset{a<b}{}\widehat{q}_{ab}x_ax_b\right],$$
(6)   
where $`Z`$ follows from the normalization of $`P(𝐱)`$. The values of the Lagrange multipliers $`\{\widehat{R}_a\}`$ and $`\{\widehat{q}_{ab}\}`$ have to determined from the constraints (On the center of mass of Ising vectors). In view of the permutation symmetry in the replica indices, $`\widehat{R}_a`$ and $`\widehat{q}_{ab}`$ must be independent of $`a`$ and $`b`$, $`\widehat{R}_a=\widehat{R},\widehat{q}_{ab}=\widehat{q}`$, rendering the evaluation of $`Z`$ very simple:

$$Z(\widehat{R},\widehat{q})=e^{n\widehat{q}/2}Dz\left[\mathrm{cosh}\left(\widehat{R}+z\sqrt{\widehat{q}}\right)\right]^n,$$
(7)   
while (On the center of mass of Ising vectors), determining $`\widehat{R}`$ and $`\widehat{q}`$, reduce to:

$`R`$ $`=`$ $`{\displaystyle \frac{1}{n}}{\displaystyle \frac{}{\widehat{R}}}\mathrm{ln}Z={\displaystyle \frac{𝑑u\mathrm{exp}\left[(u\widehat{R})^2/2\widehat{q}\right](\mathrm{cosh}u)^n\mathrm{tanh}u}{𝑑u\mathrm{exp}\left[(u\widehat{R})^2/2\widehat{q}\right](\mathrm{cosh}u)^n}}`$   
$`q`$ $`=`$ $`{\displaystyle \frac{2}{n(n1)}}{\displaystyle \frac{}{\widehat{q}}}\mathrm{ln}Z={\displaystyle \frac{𝑑u\mathrm{exp}\left[(u\widehat{R})^2/2\widehat{q}\right](\mathrm{cosh}u)^n\mathrm{tanh}^2u}{𝑑u\mathrm{exp}\left[(u\widehat{R})^2/2\widehat{q}\right](\mathrm{cosh}u)^n}}.`$ (8)   
As a result of the “replica symmetry”, we conclude from (6) that $`P(𝐱)`$ is in fact a function of $`_ax_a=ny`$. Hence $`P(y)`$ is obtained from $`P(𝐱)`$ by multiplication with a combinatorial factor, expressing the freedom to choose which $`n(1+y)/2`$ components of $`𝐱`$ are $`+1`$ and which remaining ones are $`1`$, for a given total value $`ny`$ of their sum:

$$P(y)\left(\begin{array}{c}n\\ \frac{n(1+y)}{2}\end{array}\right)\mathrm{exp}\left[n\widehat{R}y+\frac{n^2\widehat{q}y^2}{2}\right].$$
(9)   
$`y`$ can take the values $`1,1+2/n,\mathrm{},12/n,1`$, and the proportionality constant is again fixed by normalization. This result is in agreement with a direct evaluation of (3) but very different from the result for continuous components discussed in . Unfortunately, the above expression is quite complicated, especially in view of the fact that we did not succeed in solving explicitly the eqs. (On the center of mass of Ising vectors) determining the Lagrange multipliers. Concordantly, the components of $`𝐘`$ are not binary for any finite $`n`$. As an illustration, we have included in fig. 1 the results obtained by a numerical solution of (On the center of mass of Ising vectors) for the special case $`q=R`$ and several values of $`n`$. The corresponding results for the probability density for $`y`$ (or equivalently, $`B_1Y_1`$), are plotted in fig. 2.

In order to extract the asymptotic behavior for the $`n\mathrm{}`$ limit, one needs to guess the asymptotic dependence on $`n`$ of the Lagrange parameters. The correct scaling appears quite naturally in the calculations for the simpler case of vectors $`𝐉`$ with continuous components. Here, we just note by inspection that the eqs. (On the center of mass of Ising vectors) for the properly scaled Lagrange parameters $`\rho n\widehat{R}`$ and $`\gamma n\widehat{q}`$ read:

$`R`$ $`=`$ $`{\displaystyle \frac{𝑑ue^{n\varphi (u)}\mathrm{sinh}(u\rho /\gamma )\mathrm{tanh}u}{𝑑ue^{n\varphi (u)}\mathrm{cosh}(u\rho /\gamma )}}`$   
$`q`$ $`=`$ $`{\displaystyle \frac{𝑑ue^{n\varphi (u)}\mathrm{cosh}(u\rho /\gamma )\mathrm{tanh}^2u}{𝑑ue^{n\varphi (u)}\mathrm{cosh}(u\rho /\gamma )}},`$ (10)   
where

$$\varphi (u)\frac{u^2}{2\gamma }\mathrm{ln}\mathrm{cosh}u.$$
(11)   
The appearance of the hyperbolic functions of $`u\rho /\gamma `$ in eqs. (On the center of mass of Ising vectors) is due to the fact that $`\varphi `$ is even. The saddle point approximation can now, for $`n\mathrm{}`$, be applied in a straightforward manner on the $`u`$-integrations, leading to the following simple and explicit solutions for the scaled Lagrange variables:

$`\gamma `$ $`=`$ $`{\displaystyle \frac{\text{arctanh}\sqrt{q}}{\sqrt{q}}}`$   
$`\rho `$ $`=`$ $`{\displaystyle \frac{\text{arctanh}(R/\sqrt{q})}{\sqrt{q}}}.`$ (12)   
Inserting this result together with the asymptotic expression for the combinatorial factor in (9), one finally obtains the following asymptotic result for $`P(y)`$:

$`P(y)`$ $``$ $`\mathrm{exp}(\rho y)\mathrm{exp}n\left[\gamma y^2/2\mathrm{ln}\sqrt{1y^2}y\text{arctanh}y\right]`$ (13)   
$`\stackrel{n\mathrm{}}{}`$ $`{\displaystyle \frac{1}{2}}\left(1+{\displaystyle \frac{R}{\sqrt{q}}}\right)\delta (y\sqrt{q})+{\displaystyle \frac{1}{2}}\left(1{\displaystyle \frac{R}{\sqrt{q}}}\right)\delta (y+\sqrt{q}).`$   
In view of the aforementioned relation between $`y`$ and the (first) component of $`𝐘`$, the convergence of the latter to an Ising vector follows immediately.

As a first application of the above result, we turn to the case of an Ising spin system in the ferromagnetic phase. Choosing the vector $`𝐁`$ with all its components equal to $`1`$, we note that $`R`$ plays the role of the magnetization. All the spin states $`𝐉`$ are otherwise allowed, and lie on the rim of the N dimensional sphere with radius $`\sqrt{N}`$ at fixed angle $`\mathrm{arccos}R`$ with $`𝐁`$. It is thus clear that the center of mass is $`𝐁`$ itself. This trivial result is recovered from (13) by noting that $`q=R^2`$ in this case. Note that the constraint $`𝐉^a𝐉^b=Nq`$ is therefore redundant, which implies $`\widehat{q}=0`$ and $`\widehat{R}=\text{arctanh}R`$ (a result valid for any $`n`$).

A case of special symmetry is $`q=R`$. In this scenario, no macroscopic measure allows to distinguish between the symmetry breaking direction $`𝐁`$ and each of the vectors $`𝐉^a`$. The Lagrange parameters also present the symmetry $`\widehat{R}=\widehat{q}`$, which can be seen from eqs. (On the center of mass of Ising vectors).

A third case of interest is the limit $`R0`$ while $`q`$ remains finite. In this case, the $`𝐉`$-vectors lie in the subspace orthogonal to $`𝐁`$ and satisfy as single constraint the prescribed mutual overlap $`q`$. From eqs. (On the center of mass of Ising vectors) it is clear that $`\widehat{R}=0`$ is automatically satisfied, and one concludes from (13) that the center of mass of the $`𝐉`$-vectors is again an Ising vector (but its components are equally likely to be $`+1`$ or $`1`$).

It is very tempting to apply the results above to neural network learning problems  where a student perceptron $`𝐉`$ learns from examples generated by a teacher perceptron $`𝐁`$ . Indeed, so-called Gibbs learning  presents the symmetry $`q=R`$, and the interest of the center of mass is that it is, according to a simple general argument , see also , the “best” student having the largest overlap with the teacher (namely $`\sqrt{R}`$). Accordingly, $`R=0`$ and $`q0`$ are constraints satisfied by Ising vectors which solve the capacity problem . However, these are disordered systems, and the conditions on $`R`$ and $`q`$ alone do not convey all the information which is necessary to describe the constraints in $`𝐉`$ space. Therefore, even though the constraints (1) are satisfied in neural network problems, result (13) does not apply to them.

We conclude with a verification of the theoretical prediction (9) by running simulations for the mean field ferromagnetic Ising spin model, fig. 3. The Metropolis algorithm was allowed to run for a number of Monte Carlo steps per site (MCS) until thermalization was considered to be achieved. Then vectors were sampled every 5 MCS (to allow sufficient decorrelation between consecutive samplers) and summed to construct the center of mass. The small discrepancy for $`N=100`$ with the theoretical prediction is due to finite size effects. For $`N=1000`$, the results are nearly indistinguishable on the scale of the figure from the theoretical values. It is interesting to note that the hard constraints of eq. (1) are satisfied only in the thermodynamic limit. In the simulations $`R`$ (and $`q`$) are distributed with peaks whose width scales with $`N^{1/2}`$. Nonetheless the effect of these fluctuations on the resulting $`P(B_1Y_1)`$ is negligible.

\***

The authors would like to thank the referees for useful suggestions. We also acknowledge support from the FWO Vlaanderen and the Belgian IUAP program (Prime Minister’s Office).