title: Cassandra Frequent Read Write Timeouts
body: <p>I had changed whole codebase from <code>Thrift</code> to <code>CQL</code> using <code>datastax java driver 1.0.1</code> and <code>cassandra 1.2.6..</code></p>
  
  <p>with thrift I was getting frequent timeouts from start, I was not able to proceed...Adopting CQL, tables designed as per that I got success and less timeouts....</p>
  
  <p>With that I was able to insert huge data which were not working with thrift...But after a stage, data folder around 3.5GB. I am getting frequent write timeout exceptions. even I do same earlier working use case again that also throws timeout exception now. ITS RANDOM ONCE WORKED IS NOT WORKING AGAIN EVEN AFTER FRESH SETUP.</p>
  
  <p><strong>CASSADNRA SERVER LOG</strong></p>
  
  <p>this is cassandra server partial log DEBUG mode at then time I got the error : </p>
  
  <p><a href="http://pastebin.com/rW0B4MD0">http://pastebin.com/rW0B4MD0</a></p>
  
  <p>Client exception is :</p>
  
  <pre><code>Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ONE (1 replica were required but only 0 acknowledged the write)
      at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:54)
      at com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException(ResultSetFuture.java:214)
      at com.datastax.driver.core.ResultSetFuture.getUninterruptibly(ResultSetFuture.java:169)
      at com.datastax.driver.core.Session.execute(Session.java:107)
      at com.datastax.driver.core.Session.execute(Session.java:76)
  </code></pre>
  
  <p><strong>Infrastructure :</strong>
  16GB machine with 8GB heap given to cassandra, i7 processor..
  I am using SINGLE node cassandra with this yaml tweaked for timeout, everything else is default :</p>
  
  <ul>
  <li>read_request_timeout_in_ms: 30000 </li>
  <li>range_request_timeout_in_ms: 30000 </li>
  <li>write_request_timeout_in_ms: 30000 </li>
  <li>truncate_request_timeout_in_ms: 60000 </li>
  <li>request_timeout_in_ms: 30000 </li>
  </ul>
  
  <p><strong>USE CASE :</strong>
  i am running a usecase which stores Combinations(my project terminology) in cassandra....Currently testing storing 250 000 combinations with 100 parallel threads..each thread storing one combination...real case i need to support of tens of millions but that would need different hardware and multi node cluster...</p>
  
  <p>In Storing ONE combination takes around 2sec and involves:</p>
  
  <ul>
  <li>527 INSERT INTO queries</li>
  <li>506 UPDATE queries</li>
  <li>954 SELECT queries</li>
  </ul>
  
  <p>100 parallel threads parallel storing 100 combinations.</p>
  
  <p>I had found behaviour of WRITE TIMEOUTS random some time it works till 200 000 then throw timeouts AND sometimes do not work even for 10k combinations. RANDOM BEHAVIOUR.</p>