title: Convert MySql data from Latin1 to UTF8
body: <p>This is a common question which has been asked for many times before. However, I still cannot get the right answer from google.</p>
  <p>In my web app, there is a form for collecting data, the app and all data is collecting in UTF-8. However, mistakenly, the collection of the schema and the table has been set as latin1. Moreover, during the connection, &quot;SET NAMES UTF8&quot; has been used.</p>
  <p>Now some of the data in Chinese is always showing as question mark(?), no matter what conversion method I use. Query problem columns as binary also shows the data is several bytes of 3f, meaning several '?'s.</p>
  <p>If my data still be able to convert to utf-8 and shows correctly or already lost?</p>
  <p><strong>[UPDATE]</strong></p>
  <p>This is not the same question with <a href="https://stackoverflow.com/questions/6115612/how-to-convert-an-entire-mysql-database-characterset-and-collation-to-utf-8">How to convert an entire MySQL database characterset and collation to UTF-8?</a>. Because I have not just converted the entire database and table to UTF-8 but also mysqldump and re-import it into the database. However, none of them works.</p>
  <p><strong>[UPDATE 2]</strong></p>
  <p>The problem is not just about converting table charset but also need to understand UTF-8, Latin encoding system.</p>
  <p>Basic knowledge is:</p>
  <p>Latin use only 1 byte which 8 bits for storing.</p>
  <p>UTF-8 use dynamic storing system which means MAY NOT just 1 byte.</p>
  <p>Since UTF-8 encoding system needs at least 1 bit for identification, that means only 7 bits could be used for storing compare with Latin. So, if characters just need 7 bits to store, it can successfully store in Latin with UTF-8 representation. However, if data exceeds 7 bits, it will have been broken.</p>
  <p>So, such Chinese and Japaneses, it needs 2 to 3 bytes for storing, that will damage the data during storing process because the first byte in UTF-8 representation already exceed the range that Latin can store.</p>
  <p>That's why no matter how I change the charset of both the database and the table it still shows '?', because in Latin, every character that out of the range will be presented in '?', 3F in HEX.</p>