title: How to load 500M records in utf16le text encoding into Mysql 5.7 Aurora?
body: <p>I'm importing a subset of a huge CSV via workbench (MySql 5.7 Aurora)to:</p>
  <p>quickly generate the columns</p>
  <p>Then i'll load the data from S3 using LOAD DATA INFILE</p>
  <p>However, the data contains many languages and the format is utf16le</p>
  <p>When i try to import in workbench with uft16 i get</p>
  <pre><code>&quot;charmap codec can't decode&quot;..
  </code></pre>
  <p>If i extract a few rows from the CSV using powershell:</p>
  <pre><code>Get-Content .\data.csv | select -First 10 | Out-File ./test.csv
  </code></pre>
  <p>Then load test.csv using uft16 i get:</p>
  <pre><code>&quot;could not determine delimiter&quot;
  </code></pre>
  <p>Even if i set it as the default &quot;comma&quot; it doesn't work.</p>
  <p>What's the best way to handle the multi language 500M record csv for import into MySQL?</p>