title: which language to use for very large data set and lot of computation involved
body: <p>The general idea of problem is that data is arranged in following three columns in a table</p>
  
  <pre><code>"Entity" "parent entity" "value"
  A001        B001        .10
  A001        B002        .15
  A001        B003        .2
  A001        B004        .3
  A002        B002        .34
  A002        B003        .13
  ..
  ..
  ..
  A002        B111        .56
  </code></pre>
  
  <p>There is graph of entities and values can be seen as weight of directed edge from parent entity to entity. I have to calculate how many different subsets of parent entity of a particular entity are greater than .5 (say). To further calculate something (later part is easy, not complex computationally)</p>
  
  <p>The point is data is huge (Excel files says data lost :( ). which language or tool I can use? some people have suggested me SAS or STATA.</p>
  
  <p>Thanks in advance</p>