title: Cannot create JVM with -XX:+UseLargePages enabled
body: <p>I have a Java service that currently runs with a 14GB heap. I am keen to try out the <a href="http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html#PerformanceTuning" rel="noreferrer">-XX:+UseLargePages</a> option to see how this might affect the performance of the system. I have configured the OS as described <a href="http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html" rel="noreferrer">by Oracle</a> using appropriate shared memory and page values (these can also be calculated with <a href="http://www.peuss.de/node/67" rel="noreferrer">an online tool</a>).</p>
  <p>Once the OS is configured, I can see that it allocates the expected amount of memory as huge-pages. However, starting the VM with the <code>-XX:+UseLargePages</code> option set always results in one of the following errors:</p>
  <p>When <code>-Xms</code> / <code>-Xmx</code> is almost equal to the huge page allocation:</p>
  <pre><code>    Failed to reserve shared memory (errno = 28). // 'No space left on device'
  </code></pre>
  <p>When <code>-Xms</code> / <code>-Xmx</code> is less than the huge page allocation:</p>
  <pre><code>    Failed to reserve shared memory (errno = 12). // 'Out of memory'
  </code></pre>
  <p>I did try introducing some leeway - so on a 32GB system I allocated 24GB of shared memory and hugepages to use with a JVM configured with a 20GB heap, of which only 14GB is currently utilized. I also verified that the user executing the JVM did have group rights consistent with <code>/proc/sys/vm/hugetlb_shm_group</code>.</p>
  <p>Can anyone give me some pointers on where I might be going wrong and what I could try next?</p>
  <h3>Allocations/utilization:</h3>
  <ul>
  <li><code>-Xms</code> / <code>-Xmx</code> - 20GB</li>
  <li>Utilized heap - 14GB</li>
  <li><code>/proc/sys/kernel/shmmax</code> - 25769803776 (24GB)</li>
  <li><code>/proc/sys/vm/nr_hugepages</code> - 12288</li>
  </ul>
  <h3>Environment:</h3>
  <ul>
  <li>System memory - 32GB</li>
  <li>System page size - 2048KB</li>
  <li>debian 2.6.26-2-amd64</li>
  <li>Sun JVM 1.6.0_20-b02</li>
  </ul>
  <h1>Solution</h1>
  <p>Thanks to <a href="https://stackoverflow.com/users/851677/jfgagne">@jfgagne</a> for providing an answer that lead to a solution. In addition to the <code> /proc/sys/kernel/shmall</code> setting (specified as 4KB pages), I had to add entries to <code>/etc/security/limits.conf</code> as described on <a href="http://www.peuss.de/node/67" rel="noreferrer">Thomas' blog</a>. However, as my application is started using <code>jsvc</code> I also had to duplicate the settings for the root user (note that the limits are specified in KB):</p>
  <pre><code>    root       soft memlock 25165824
      root       hard memlock 25165824
      pellegrino soft memlock 25165824
      pellegrino hard memlock 25165824
  </code></pre>
  <p>It's also worth mentioning that settings could be tested quickly by starting the JVM with the <code>-version</code> argument:</p>
  <pre><code>    java -XX:+UseLargePages -Xmx20g -version
  </code></pre>