title: How to filter by date range in Spark SQL
body: <p>I'm trying to filter the date range from the following data using Data bricks, which returns null as response.
  My csv data looks like:</p>
  
  <pre><code>ID, Desc, Week_Ending_Date
  100, AAA, 13-06-2015
  101, BBB, 11-07-2015
  102, CCC, 15-08-2015
  103, DDD, 05-09-2015
  100, AAA, 29-08-2015
  100, AAA, 22-08-2015
  </code></pre>
  
  <p>My query is:</p>
  
  <pre><code>df.select(df("ID"), date_format(df("Week_Ending_Date"), "yyyy-MM-dd"))
  .filter(date_format(df("Week_Ending_Date"), "yyyy-MM-  dd").between("2015-07-05", "2015-09-02"))
  </code></pre>
  
  <p>Any help is much appreciated.</p>