title: How to read partitioned table from BigQuery to Spark dataframe (in PySpark)
body: <p>I have a BQ table and it's partitioned by the default <code>_PARTITIONTIME</code>. I want to read one of its partitions to Spark dataframe (PySpark). However, the spark.read API doesn't seem to recognize the partition column. Below is the code (which doesn't work):</p>
  
  <pre><code>table = 'myProject.myDataset.table'
  
  df = spark.read.format('bigquery').option('table', table).load()
  
  df_pt = df.filter("_PARTITIONTIME = TIMESTAMP('2019-01-30')")
  </code></pre>
  
  <p>The partition is quite large so I'm not able to read as a pandas dataframe.</p>
  
  <p>Thank you very much.</p>