title: mmap() vs. reading blocks
body: <p>I'm working on a program that will be processing files that could potentially be 100GB or more in size. The files contain sets of variable length records. I've got a first implementation up and running and am now looking towards improving performance, particularly at doing I/O more efficiently since the input file gets scanned many times.</p>
  <p>Is there a rule of thumb for using <code>mmap()</code> versus reading in blocks via C++'s <code>fstream</code> library? What I'd like to do is read large blocks from disk into a buffer, process complete records from the buffer, and then read more.</p>
  <p>The <code>mmap()</code> code could potentially get very messy since <code>mmap</code>'d blocks need to lie on page sized boundaries (my understanding) and records could potentially lie across page boundaries. With <code>fstream</code>s, I can just seek to the start of a record and begin reading again, since we're not limited to reading blocks that lie on page sized boundaries.</p>
  <p>How can I decide between these two options without actually writing up a complete implementation first? Any rules of thumb (e.g., <code>mmap()</code> is 2x faster) or simple tests?</p>