title: Spark - how to skip or ignore empty gzip files when reading
body: <p>I have a couple of hundred folders with some thousands of gzipped text files each in s3 and I'm trying to read them into a dataframe with <code>spark.read.csv()</code>. </p>
  
  <p>Among the files, there are some with zero length, resulting in the error:</p>
  
  <blockquote>
    <p>java.io.EOFException: Unexpected end of input stream</p>
  </blockquote>
  
  <p>Code:</p>
  
  <pre><code>df = spark.read.csv('s3n://my-bucket/folder*/logfiles*.log.gz',sep='\t',schema=schema)
  </code></pre>
  
  <p>I've tried setting the <code>mode</code> to <code>DROPMALFORMED</code> and reading with <code>sc.textFile()</code> but no luck.</p>
  
  <p>What's the best way to handle empty or broken gzip files?</p>