title: Slow index speed of Elasticsearch
body: <p>We deployed ES 2.0 on 3 EC2 c4.4xlarge(16 cores, 32gb memory) nodes, allocating 16G for ES, attached 500GB with io1/4000 IOPS on each.</p>
  
  <p><strong>Problem</strong> : We are expecting great performance from this hardware config, however <strong><em>a very slow indexing speed is observed</em></strong>.</p>
  
  <p>Our document is about 10-50k in size, we are using Java transport client to insert. The speed was alright for the first 50,000 at roughly 1000/second, and dramatically slow down to 100-200/second.</p>
  
  <p>In the meanwhile we are looking at the low resource consumption:</p>
  
  <ol>
  <li>CPU is about 1-20% only (16 Core CPU)</li>
  <li>IO write is about 4-10Mb/second only</li>
  <li>Memory consumption is about 20-30% only</li>
  </ol>
  
  <p><strong>Requirements</strong> :So I cannot understand why it is so slow while all the recourses are so free, <strong><em>what can I do to enhance the efficiency?</em></strong> Thanks.</p>
  
  <p>Here is the config file we are using:</p>
  
  <pre><code>cluster.name: {{ env }}-{{ app }}
  path.data: /data/es
  path.logs: /data/es-logs
  network.host: 0.0.0.0
  discovery.zen.ping.unicast.hosts: ["xxxx"]
  bootstrap.mlockall: true
  threadpool.search.queue_size: 300
  threadpool.index.type: fixed
  threadpool.index.size: 16
  threadpool.index.queue_size: 250000
  index.refresh_interval: 1s
  index.translog.flush_threshold_ops: 50000
  indices.memory.index_buffer_size: 30%
  indices.memory.min_shard_index_buffer_size: 12mb
  indices.memory.min_index_buffer_size: 96mb
  script.inline: on
  script.indexed: on
  http.cors.enabled: true
  http.cors.allow-origin: /https?:\/\/localhost(:[0-9]+)?/
  </code></pre>
  
  <p>Here is htop and iostat while running the job:
  <a href="https://i.stack.imgur.com/aCgzl.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/aCgzl.jpg" alt="htop"></a></p>
  
  <p><a href="https://i.stack.imgur.com/jVfg1.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/jVfg1.jpg" alt="iostat"></a></p>