title: improving performance of mysql load data infile
body: <p>I'm trying to bulk load around 12m records into a InnoDB table in a (local) mysql using LOAD DATA INFILE (from  CSV) and finding it's taking a very long time to complete.</p>
  
  <p>The primary key type is UUID and the keys are unsorted in the data files.</p>
  
  <p>I've split the data file into files containing 100000 records and import it as:</p>
  
  <pre><code>mysql -e 'ALTER TABLE customer DISABLE KEYS;'
  for file in *.csv
      mysql -e "SET sql_log_bin=0;SET FOREIGN_KEY_CHECKS=0; SET UNIQUE_CHECKS=0;
      SET AUTOCOMMIT=0;LOAD DATA INFILE '${file}' INTO TABLE table 
      FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';  COMMIT"
  </code></pre>
  
  <p>This works fine for the first few hundred thousand records but then the insert time for each subsequent load seems to keep growing (from around 7 seconds to around 2 minutes per load before I killed it.)</p>
  
  <p>I'm running on a machine with 8GB RAM and have set the InnoDB parameters to:</p>
  
  <pre><code>innodb_buffer_pool_size =1024M
  innodb_additional_mem_pool_size =512M
  innodb_log_file_size = 256M
  innodb_log_buffer_size = 256M
  </code></pre>
  
  <p>I've also tried loading a single CSV containing all rows with no luck - this ran in excess of 2 hours before I killed it.</p>
  
  <p>Is there anything else that could speed this up as this seems like an excessive time to only load 12m records? </p>