title: Force JVM to do all IO without page cache (e.g. O_DIRECT)
body: <p>I'm doing some benchmarks of an application written in Java. It is very important for the experiments that the results are not influenced by the page cache (I'm using linux)</p>
  
  <p>So the best way to avoid the page cache is using O_DIRECT whenever a file is opened. I therefore changed the respective code in the sourcecode of the jre.</p>
  
  <p>My approach works perfectly for everything that goes through the <code>FileOutputStream</code> (e.g. writing), but it does not work for <code>FileInputStream</code> (e.g. reading).</p>
  
  <p>When adding O_DIRECT to the open-call of <code>FileInputStream</code>, the JVM is unable to load any classes:</p>
  
  <pre><code>Error: Could not find or load main class perf.TestDirectIO
  </code></pre>
  
  <p>This error is not a classpath issue, since I can fix it just by using a "unhacked" JVM.</p>
  
  <p>So there seems to be an issue with opening files.</p>
  
  <p>I'm very happy about any advice on how to fix the issue.</p>
  
  <p>If anyone ever wants to do something similar, I've <a href="http://rmnotes.wordpress.com/2013/03/05/hacking-o_direct-into-the-java-jvm-or-how-to-build-the-jvm-on-arch-linux/" rel="noreferrer">documented the whole hack in my blog</a>.</p>
  
  <hr>
  
  <p>As a reference, these are the changes on the JVM code I did so:</p>
  
  <p><code>jdk/src/share/native/java/io/FileInputStream.c</code>:</p>
  
  <pre><code> @@ -58,7 +60,8 @@
   JNIEXPORT void JNICALL
   Java_java_io_FileInputStream_open(JNIEnv *env, jobject this, jstring path) {
  -    fileOpen(env, this, path, fis_fd, O_RDONLY);
  +    fileOpen(env, this, path, fis_fd, O_RDONLY | O_DIRECT); // this is the change that causes all the problems
   }
  </code></pre>
  
  <p>This change works:
  <code>jdk/src/solaris/native/java/io/FileOutputStream_md.c</code>:</p>
  
  <pre><code>@@ -55,8 +55,10 @@
   JNIEXPORT void JNICALL
   Java_java_io_FileOutputStream_open(JNIEnv *env, jobject this,
                                      jstring path, jboolean append) {
       fileOpen(env, this, path, fos_fd,
  -             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
  +             O_WRONLY | O_DIRECT | O_CREAT | (append ? O_APPEND : O_TRUNC));
   }
  </code></pre>
  
  <p>I also changed the hotspot jre to ensure that the memory is aligned (thats a requirement for O_DIRECT)
  <code>hotspot/src/share/vm/runtime/os.cpp</code>:</p>
  
  <pre><code>+# include &lt;mm_malloc.h&gt;
  ...
  -  u_char* ptr = (u_char*)::malloc(size + space_before + space_after);
  +  u_char* ptr = (u_char*)::_mm_malloc(size + space_before + space_after,512);
  </code></pre>