title: How to read multiple gzipped files from S3 into a single RDD?
body: <p>I have many gzipped files stored on S3 which are organized by project and hour per day, the pattern of the paths of the files is as:</p>
  
  <pre><code>s3://&lt;bucket&gt;/project1/20141201/logtype1/logtype1.0000.gz
  s3://&lt;bucket&gt;/project1/20141201/logtype1/logtype1.0100.gz
  ....
  s3://&lt;bucket&gt;/project1/20141201/logtype1/logtype1.2300.gz
  </code></pre>
  
  <p>Since the data should be analyzed on a daily basis, I have to download and decompress the files belongs to a specific day, then assemble the content as a single RDD.</p>
  
  <p>There should be several ways can do this, but I would like to know the best practice for Spark.</p>
  
  <p>Thanks in advance.</p>