title: Spark on AWS EMR - Dynamic partition overwrite S3 / Glue
body: <p>I'm using pyspark from an AWS EMR notebook and would like to overwrite a single partition when saving a table. Normally can do this with</p>
  <pre><code>df.write.mode('overwrite')\
              .option(&quot;partitionOverwriteMode&quot;, &quot;dynamic&quot;)\
              .insertInto('table')
  </code></pre>
  <p>However this doesn't work when writing to S3. Is there a way to only overwrite the files in that S3 partition, as well as in the spark metadata?</p>
  <p>Note: I am using Glue as the spark metadata</p>