title: Using windowing functions in Spark
body: <p>I am trying to use rowNumber in Spark data frames. My queries are working as expected in Spark shell. But when i write them out in eclipse and compile a jar, i am facing an error </p>
  
  <pre><code> 16/03/23 05:52:43 ERROR ApplicationMaster: User class threw exception:org.apache.spark.sql.AnalysisException: Could not resolve window function 'row_number'. Note that, using window functions currently requires a HiveContext;
  org.apache.spark.sql.AnalysisException: Could not resolve window function 'row_number'. Note that, using window functions currently requires a HiveContext;
  </code></pre>
  
  <p>My queries</p>
  
  <pre class="lang-scala prettyprint-override"><code>import org.apache.spark.sql.functions.{rowNumber, max, broadcast}
  import org.apache.spark.sql.expressions.Window
  val w = Window.partitionBy($"id").orderBy($"value".desc)
  
  val dfTop = df.withColumn("rn", rowNumber.over(w)).where($"rn" &lt;= 3).drop("rn")
  </code></pre>
  
  <p>I am not using HiveContext while running the queries in Spark shell. Not sure why it is returning an error when i run the same as a jar file. And also I am running the scripts on Spark 1.6.0 if that helps. Did anyone face similar issue?</p>