title: Copy a file in a sane, safe and efficient way
body: <p>I search for a good way to copy a file (binary or text). I've written several samples, everyone works. But I want hear the opinion of seasoned programmers.</p>
  
  <p>I missing good examples and search a way which works with C++.</p>
  
  <p><strong>ANSI-C-WAY</strong></p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;cstdio&gt;    // fopen, fclose, fread, fwrite, BUFSIZ
  #include &lt;ctime&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      // BUFSIZE default is 8192 bytes
      // BUFSIZE of 1 means one chareter at time
      // good values should fit to blocksize, like 1024 or 4096
      // higher values reduce number of system calls
      // size_t BUFFER_SIZE = 4096;
  
      char buf[BUFSIZ];
      size_t size;
  
      FILE* source = fopen("from.ogv", "rb");
      FILE* dest = fopen("to.ogv", "wb");
  
      // clean and more secure
      // feof(FILE* stream) returns non-zero if the end of file indicator for stream is set
  
      while (size = fread(buf, 1, BUFSIZ, source)) {
          fwrite(buf, 1, size, dest);
      }
  
      fclose(source);
      fclose(dest);
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt; end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>POSIX-WAY</strong> (K&amp;R use this in "The C programming language", more low-level)</p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;fcntl.h&gt;   // open
  #include &lt;unistd.h&gt;  // read, write, close
  #include &lt;cstdio&gt;    // BUFSIZ
  #include &lt;ctime&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      // BUFSIZE defaults to 8192
      // BUFSIZE of 1 means one chareter at time
      // good values should fit to blocksize, like 1024 or 4096
      // higher values reduce number of system calls
      // size_t BUFFER_SIZE = 4096;
  
      char buf[BUFSIZ];
      size_t size;
  
      int source = open("from.ogv", O_RDONLY, 0);
      int dest = open("to.ogv", O_WRONLY | O_CREAT /*| O_TRUNC/**/, 0644);
  
      while ((size = read(source, buf, BUFSIZ)) &gt; 0) {
          write(dest, buf, size);
      }
  
      close(source);
      close(dest);
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt; end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>KISS-C++-Streambuffer-WAY</strong></p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;fstream&gt;
  #include &lt;ctime&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      ifstream source("from.ogv", ios::binary);
      ofstream dest("to.ogv", ios::binary);
  
      dest &lt;&lt; source.rdbuf();
  
      source.close();
      dest.close();
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt;  end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>COPY-ALGORITHM-C++-WAY</strong></p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;fstream&gt;
  #include &lt;ctime&gt;
  #include &lt;algorithm&gt;
  #include &lt;iterator&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      ifstream source("from.ogv", ios::binary);
      ofstream dest("to.ogv", ios::binary);
  
      istreambuf_iterator&lt;char&gt; begin_source(source);
      istreambuf_iterator&lt;char&gt; end_source;
      ostreambuf_iterator&lt;char&gt; begin_dest(dest); 
      copy(begin_source, end_source, begin_dest);
  
      source.close();
      dest.close();
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt;  end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>OWN-BUFFER-C++-WAY</strong></p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;fstream&gt;
  #include &lt;ctime&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      ifstream source("from.ogv", ios::binary);
      ofstream dest("to.ogv", ios::binary);
  
      // file size
      source.seekg(0, ios::end);
      ifstream::pos_type size = source.tellg();
      source.seekg(0);
      // allocate memory for buffer
      char* buffer = new char[size];
  
      // copy file    
      source.read(buffer, size);
      dest.write(buffer, size);
  
      // clean up
      delete[] buffer;
      source.close();
      dest.close();
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt;  end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>LINUX-WAY</strong> // requires kernel >= 2.6.33</p>
  
  <pre><code>#include &lt;iostream&gt;
  #include &lt;sys/sendfile.h&gt;  // sendfile
  #include &lt;fcntl.h&gt;         // open
  #include &lt;unistd.h&gt;        // close
  #include &lt;sys/stat.h&gt;      // fstat
  #include &lt;sys/types.h&gt;     // fstat
  #include &lt;ctime&gt;
  using namespace std;
  
  int main() {
      clock_t start, end;
      start = clock();
  
      int source = open("from.ogv", O_RDONLY, 0);
      int dest = open("to.ogv", O_WRONLY | O_CREAT /*| O_TRUNC/**/, 0644);
  
      // struct required, rationale: function stat() exists also
      struct stat stat_source;
      fstat(source, &amp;stat_source);
  
      sendfile(dest, source, 0, stat_source.st_size);
  
      close(source);
      close(dest);
  
      end = clock();
  
      cout &lt;&lt; "CLOCKS_PER_SEC " &lt;&lt; CLOCKS_PER_SEC &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME START " &lt;&lt; start &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END " &lt;&lt; end &lt;&lt; "\n";
      cout &lt;&lt; "CPU-TIME END - START " &lt;&lt;  end - start &lt;&lt; "\n";
      cout &lt;&lt; "TIME(SEC) " &lt;&lt; static_cast&lt;double&gt;(end - start) / CLOCKS_PER_SEC &lt;&lt; "\n";
  
      return 0;
  }
  </code></pre>
  
  <p><strong>Environment</strong>  </p>
  
  <ul>
  <li>GNU/LINUX (Archlinux)</li>
  <li>Kernel 3.3</li>
  <li>GLIBC-2.15, LIBSTDC++ 4.7 (GCC-LIBS), GCC 4.7, Coreutils 8.16</li>
  <li>Using RUNLEVEL 3 (Multiuser, Network, Terminal, no GUI)</li>
  <li>INTEL SSD-Postville 80 GB, filled up to 50%  </li>
  <li>Copy a 270 MB OGG-VIDEO-FILE  </li>
  </ul>
  
  <p><strong>Steps to reproduce</strong> </p>
  
  <pre><code> 1. $ rm from.ogg
   2. $ reboot                           # kernel and filesystem buffers are in regular
   3. $ (time ./program) &amp;&gt;&gt; report.txt  # executes program, redirects output of program and append to file
   4. $ sha256sum *.ogv                  # checksum
   5. $ rm to.ogg                        # remove copy, but no sync, kernel and fileystem buffers are used
   6. $ (time ./program) &amp;&gt;&gt; report.txt  # executes program, redirects output of program and append to file
  </code></pre>
  
  <p><strong>Results (CPU TIME used)</strong>  </p>
  
  <pre><code>Program  Description                 UNBUFFERED|BUFFERED
  ANSI C   (fread/frwite)                 490,000|260,000  
  POSIX    (K&amp;R, read/write)              450,000|230,000  
  FSTREAM  (KISS, Streambuffer)           500,000|270,000 
  FSTREAM  (Algorithm, copy)              500,000|270,000
  FSTREAM  (OWN-BUFFER)                   500,000|340,000  
  SENDFILE (native LINUX, sendfile)       410,000|200,000  
  </code></pre>
  
  <p>Filesize doesn't change.<br>
  sha256sum print the same results.<br>
  The video file is still playable.  </p>
  
  <p><strong>Questions</strong></p>
  
  <ul>
  <li>What method would you prefer?</li>
  <li>Do you know better solutions?</li>
  <li>Do you see any mistakes in my code?</li>
  <li><p>Do you know a reason to avoid a solution?</p></li>
  <li><p>FSTREAM (KISS, Streambuffer)<br>
  I really like this one, because it is really short and simple. As far is I know the operator &lt;&lt; is overloaded for rdbuf() and doesn't convert anything. Correct?</p></li>
  </ul>
  
  <p>Thanks</p>
  
  <p><strong>Update 1</strong><br>
  I changed the source in all samples in that way, that the open and close of the file descriptors is include in the measurement of <em>clock()</em>. Their are no other significant changes in the source code. The results doesn't changed! I also used <em>time</em> to double-check my results.  </p>
  
  <p><strong>Update 2</strong><br>
  ANSI C sample changed: The condition of the <em>while-loop</em> doesn't call any longer <em>feof()</em> instead I moved <em>fread()</em> into the condition. It looks like, the code runs now 10,000 clocks faster.  </p>
  
  <p>Measurement changed: The former results were always buffered, because I repeated the old command line <em>rm to.ogv &amp;&amp; sync &amp;&amp; time ./program</em> for each program a few times. Now I reboot the system for every program. The unbuffered results are new and show no surprise. The unbuffered results didn't changed really.</p>
  
  <p>If i don't delete the old copy, the programs react different. Overwriting a existing file <em>buffered</em> is faster with POSIX and SENDFILE, all other programs are slower. Maybe the options <em>truncate</em> or <em>create</em> have a impact on this behaviour. But overwriting existing files with the same copy is not a real world use-case.  </p>
  
  <p>Performing the copy with <em>cp</em> takes 0.44 seconds unbuffered und 0.30 seconds buffered. So <em>cp</em> is a little bit slower than the POSIX sample. Looks fine for me.  </p>
  
  <p>Maybe I add also samples and results of <em>mmap()</em> and <em><code>copy_file()</code></em> from boost::filesystem.</p>
  
  <p><strong>Update 3</strong><br>
  I've put this also on a blog page and extended it a little bit. Including <em>splice()</em>, which is a low-level function from the Linux kernel. Maybe more samples with Java will follow.
  <a href="http://www.ttyhoney.com/blog/?page_id=69">http://www.ttyhoney.com/blog/?page_id=69</a></p>