title: How to handle user input of invalid UTF-8 characters
body: <p>I'm looking for a general strategy/advice on how to handle invalid UTF-8 input from users.</p>
  <p>Even though my web application uses UTF-8, somehow some users enter invalid characters. This causes errors in PHP's <a href="http://us2.php.net/json_encode" rel="nofollow noreferrer">json_encode()</a> and overall seems like a bad idea to have around.</p>
  <p><a href="http://www.w3.org/International/questions/qa-forms-utf-8" rel="nofollow noreferrer">W3C I18N FAQ: Multilingual Forms</a> says &quot;If non-UTF-8 data is received, an error message should be sent back.&quot;.</p>
  <ul>
  <li>How exactly should this be practically done, throughout a site with dozens of different places where data can be input?</li>
  <li>How do you present the error in a helpful way to the user?</li>
  <li>How do you temporarily store and display bad form data so the user doesn't lose all their text? Strip bad characters? Use a replacement character, and how?</li>
  <li>For existing data in the database, when invalid UTF-8 data is detected, should I try to convert it and save it back (how? <a href="http://us3.php.net/utf8_encode" rel="nofollow noreferrer">utf8_encode</a>()? <a href="http://php.net/manual/en/function.mb-convert-encoding.php" rel="nofollow noreferrer">mb_convert_encoding()</a>?), or leave as-is in the database but doing something (what?) before json_encode()?</li>
  </ul>
  <p>I'm very familiar with the mbstring extension and am not asking &quot;how does UTF-8 work in PHP?&quot;. I'd like advice from people with experience in real-world situations how they've handled this.</p>
  <p>As part of the solution, I'd really like to see a <em>fast</em> method to convert invalid characters to <a href="https://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280" rel="nofollow noreferrer">U+FFFD</a>.</p>