title: Multibyte-safe fread in PHP
body: <p>I have a file that's too large to fit into memory, from which I need to strip certain characters (control characters to be precise). My current function looks like this:</p>
  
  <pre><code>$old = fopen($file, 'r');
  $new = fopen($tmpFile, 'w');
  
  while (!feof($old)) {
      fwrite($new, preg_replace('/[^\P{Cc}\t\r\n]/u', '', fgets($old)));
  }
  
  rename($tmpFile, $file);
  </code></pre>
  
  <p>This works fine in most cases. A possible problem though is that <code>fgets</code> reads an entire line. Some files I process are literally huge one-liners, which would still cause memory issues.</p>
  
  <p>This can be fixed using <code>fread</code>, with a chunk size of say 8192. However now the text I feed <code>preg_replace</code> could be cut-off multibyte characters.</p>
  
  <p>I've been thinking how I can <code>fread</code> while preserving multibyte characters, but I haven't found a good solution yet. Any help would be awesome.</p>
  
  <h2>Possible solution</h2>
  
  <p>While I've solved the problem in a different way, I'm still curious about my original question: how to do a mb-safe <code>fread</code>? I think a function like this could work:</p>
  
  <ol>
  <li>Read a chunk of bytes with <code>fread</code></li>
  <li>Inspect the last byte, check if it's part of a multibyte sequence. If not, stop here.</li>
  <li>Keep reading bytes until the last byte is not part of a multibyte sequence, or ends the current sequence.</li>
  </ol>
  
  <p>Step 2 could probably use some logic <a href="https://github.com/Behat/Transliterator/blob/master/src/Behat/Transliterator/Transliterator.php#L34-L51" rel="nofollow">like this</a>, but I'm not that experienced with unicode that I know how to.</p>