title: fetch data from hive table into spark and perform join on RDDs
body: <p>I have two tables in hive/impala. I want to fetch the data from the table into spark as rdds and perform say a join operation.</p>
  
  <p>I do not want to directly pass the join query in my hive context. This is just an example.
  I have more use cases that are not possible by a standard HiveQL.
  How do I fetch all rows, access the columns and perform transformation.</p>
  
  <p>Suppose I have two rdds:</p>
  
  <pre><code>val table1 =  hiveContext.hql("select * from tem1")
  
  val table2 =  hiveContext.hql("select * from tem2")
  </code></pre>
  
  <p>I want to perform a join on the rdds on a column called "account_id"</p>
  
  <p>Ideally I want to do something like this using the rdds using spark shell.</p>
  
  <pre><code>select * from tem1 join tem2 on tem1.account_id=tem2.account_id; 
  </code></pre>