title: Java memory usage much higher than heap+nonheap
body: <p>I have a Java application (oracle JDK 8u191) with SpringBoot version 2.1.1.RELEASE and I'm experiencing a weird memory usage (don't thinks this is related to spring).</p>
  
  <p>The sum of the HEAP and NONHEAP values provided by the application is much lower the real memory usage reported by Docker.</p>
  
  <p>I know that Java has others memory areas than HEAP and NONHEAP, but I don't know how to measure them and if the current behavior is normal.</p>
  
  <p>Curently my applications is reporting the following (after 3 days running in production):</p>
  
  <pre><code>           Init       Used       Committed       Max       
  HEAP:      250M       79M        250M            800M      
  NONHEAP:   2M         192M       201M            0M     
  </code></pre>
  
  <p>The actuator endpoint <code>/metrics</code> for springBoot reports the same memory usage.</p>
  
  <p>But the docker stats commands reports this (inside the container <code>smem</code> reports the same value for the java process):</p>
  
  <pre><code>MEM USAGE / LIMIT
  750.2MiB / 1.172GiB
  </code></pre>
  
  <p>The difference between the heap+nonheap and the current used memory are <code>299,2mb</code>.</p>
  
  <p>I was not expecting this 300mb the first time we put this service in production it caused a lot of OOM until we found the rigth configuration.</p>
  
  <p>This "unregistered memory usage" seems to only go up and never shrink, even after GC.</p>
  
  <p>If I restart my aplication the difference between the heap+nonheap withe the real usage is about 100mb.</p>
  
  <p>My Dockerfile starts the application this way:</p>
  
  <p><code>ENTRYPOINT ["java", "-Xms250m", "-Xmx800m", "-XX:+UseG1GC", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/app.jar"]</code></p>
  
  <p>I also have a scheduled rotine that runs <code>System.gc()</code> every hour to shrink back the commited memory because this service have small spikes over the day and I want unused memory to be free for another services.</p>
  
  <p>I want to know if this behavior is normal. And how can I measure what is using so much memory? visualvm and jconsole only show the heap and nonheap. Seems there is no metrics in springBoot actuator to measure this too.</p>
  
  <p>How much memory should I left free in my container for this type of usage? I couldn't find any helpful article about it.</p>
  
  <p><strong>Edit1:</strong></p>
  
  <p>After 7 days, the difference between (heap+nonheap)-total became 550mb, leaving my container without any memory left.</p>
  
  <p><strong>Edit2:</strong></p>
  
  <p>Things starting making sense after digging into this articles:</p>
  
  <p><a href="https://stackoverflow.com/questions/47591343/java-process-memory-usage-deviation-from-jcmd-committed-memory?rq=1">Java process memory usage deviation from jcmd committed memory</a></p>
  
  <p><a href="https://stackoverflow.com/questions/44284911/where-do-these-java-native-memory-allocated-from">Where do these java native memory allocated from?</a></p>
  
  <p><a href="https://stackoverflow.com/questions/39684464/java-process-memory-usage-jcmd-vs-pmap">Java process memory usage (jcmd vs pmap)</a></p>
  
  <p>And indeed we are using a third party library with reports of memory leaks: <code>compile("axis:axis-wsdl4j:1.5.1")</code></p>
  
  <p>I'll try upgrade to Axis2 and will post the results here.</p>