title: Apache Spark reads for S3: can't pickle thread.lock objects
body: <p>So I want my Spark App to read some text from Amazon's S3. I Wrote the following simple script:</p>
  
  <pre class="lang-py prettyprint-override"><code>import boto3
  s3_client = boto3.client('s3')
  text_keys = ["key1.txt", "key2.txt"]
  data = sc.parallelize(text_keys).flatMap(lambda key: s3_client.get_object(Bucket="my_bucket", Key=key)['Body'].read().decode('utf-8'))
  </code></pre>
  
  <p>When I do <code>data.collect</code> I get the following error:</p>
  
  <pre><code>TypeError: can't pickle thread.lock objects
  </code></pre>
  
  <p>and I don't seem to find any help online. Have perhaps someone managed to solve the above?</p>