title: Converting JDBC ResultSet to Spark RDD/DataFrame
body: <p>I was trying to convert JDBC ResultSet to Spark RDD and was looking for an efficient way to do that using parallelism feature of Spark.</p>
  
  <p>Below is what i have implemented as per this <a href="https://stackoverflow.com/a/32073423/6064131">https://stackoverflow.com/a/32073423/6064131</a></p>
  
  <pre><code>val rs:ResultSet = stmt .getResultSet
  val colCount = rs.getMetaData.getColumnCount
  
  def getRowFromResultSet(resultSet: ResultSet): String ={
    var i:Int = 1
    var rowStr=""
    while(i&lt;=colCount){
      rowStr=rowStr+resultSet.getString(i)+delim
      i+=1
    }
    rowStr
  }
  
  val resultSetList = Iterator.continually((rs.next(), rs)).takeWhile(_._1).map(r =&gt; {
    getRowFromResultSet(r._2) // (ResultSet) =&gt; (spark.sql.Row)
  }).toList
  
  val x = sc.parallelize(resultSetList)
  </code></pre>
  
  <p>Now the main issue is it is taking more time and i understand all dataset is pulled through one needle eye.But is there any better way to achieve this?</p>
  
  <p>Some might be wondering why i am not using inbuilt feature sqlContext.read.format to achieve this, the reason is Spark wraps a "SELECT * FROM ( )" around the query which is creating issue with complex queries.
  Please refer the link for details <a href="https://stackoverflow.com/questions/45593080/issue-with-with-clause-with-cloudera-jdbc-driver-for-impala-returning-column-n">Issue with WITH clause with Cloudera JDBC Driver for Impala - Returning column name instead of actual Data</a></p>