title: RDD split and do aggregation on new RDDs
body: <p>I have an RDD of <code>(String,String,Int)</code>.</p>
  
  <ol>
  <li>I want to reduce it based on the first two strings</li>
  <li>And Then based on the first String I want to group the (String,Int) and sort them </li>
  <li>After sorting I need to group them into small groups each containing n elements.</li>
  </ol>
  
  <p>I have done the code below. The problem is the number of elements in the step 2 is very large for a single key 
  and the <code>reduceByKey(x++y)</code> takes a lot of time. </p>
  
  <pre><code>//Input
  val data = Array(
    ("c1","a1",1), ("c1","b1",1), ("c2","a1",1),("c1","a2",1), ("c1","b2",1), 
    ("c2","a2",1), ("c1","a1",1), ("c1","b1",1), ("c2","a1",1))
  
  val rdd = sc.parallelize(data)
  val r1 = rdd.map(x =&gt; ((x._1, x._2), (x._3)))
  val r2 = r1.reduceByKey((x, y) =&gt; x + y ).map(x =&gt; ((x._1._1), (x._1._2, x._2)))
  
  // This is taking long time.
  val r3 = r2.mapValues(x =&gt; ArrayBuffer(x)).reduceByKey((x, y) =&gt; x ++ y) 
  
  // from the list I will be doing grouping.
  val r4 = r3.map(x =&gt; (x._1 , x._2.toList.sorted.grouped(2).toList)) 
  </code></pre>
  
  <p>Problem is the "c1" has lot of unique entries like b1 ,b2....million and <code>reduceByKey</code> is killing time because all the values are going to single node.
  Is there a way to achieve this more efficiently?</p>
  
  <pre><code>// output
   Array((c1,List(List((a1,2), (a2,1)), List((b1,2), (b2,1)))), (c2,List(List((a1,2), (a2,1)))))
  </code></pre>