title: Mysql or Spark Processing of 400gb data
body: <p>If I use spark in my case, based on block and cores will it be useful ?</p>
  
  <p>I have 400 GB of data in single table i.e. <code>User_events</code> with multiple columns in MySQL. This table stores all user events from application. Indexes are there on required columns. I have an user interface where user can try different permutation and combination of fields under <code>user_events</code></p>
  
  <p>Currently I am facing the performance issues where query either takes 15/20 seconds or even longer or times out.</p>
  
  <p>I have gone through couple of Spark tutorial but I am not sure if it can help here. Per mine understanding from spark,  </p>
  
  <ol>
  <li>First Spark has to bring all the data in memory. Bring 100 M record on netwok will be costly operation  and I will be needing big memory for the
  same. Isn't it ?</li>
  <li>Once data in memory, Spark can distribute the data among partition based on cores and input data size. Then it can filter the data on each partition
  in parallel. Here Spark can be beneficial as it can do the parallel operation while MySQL will be sequential. Is that correct ?</li>
  </ol>
  
  <p>Is my understanding correct ?</p>