title: Spark RDD - which one of the duplicate record distinct operation will pick
body: <p>I have a RDD</p>
  
  <pre><code> JavaPairRDD&lt;String,Customer&gt; RDD1
  </code></pre>
  
  <p>Which has one record </p>
  
  <pre><code>cust_id, first_name, lastname
  1      "rahul"      "koshaley"
  </code></pre>
  
  <p>and <code>JavaPairRDD&lt;String,Customer&gt; RDD2</code></p>
  
  <p>again which has one record</p>
  
  <pre><code> cust_id , first_name , last_name 
     1        "rahul"          ""
  </code></pre>
  
  <p>when I do union <code>JavaPairRDD&lt;String,Customer&gt; unionRDD  =  RDD1.union(RDD2);</code></p>
  
  <p>The union operation gives me 2 records</p>
  
  <pre><code>1)  1 , "rahul" , "koshaley"
  2)  1 , "rahul" , ""
  </code></pre>
  
  <p>Now when I do distinct on unionRDD ie </p>
  
  <pre><code>JavaPairRDD&lt;String,Customer&gt; distinct = unionRDD.distinct();
  </code></pre>
  
  <p>will the resulting RDD distinct give me output as </p>
  
  <pre><code> 1 , "rahul" , "koshaley" or
  
   1 , "rahul" , ""
  </code></pre>
  
  <p>I want the output RDD to contain the record which has all the values 
  <code>ie 1 , "rahul" , "koshaley"</code>   !</p>
  
  <p>EDIT -> THIS QUESTION IS NOT DUPLICATE as I NEED TO KNOW WHICH ONE OF THE DUPLICATE RECORDS WILL SPARK PICK AFTER DISTINCT OPERATION.</p>