title: EMR - Pyspark error -Container exited with a non-zero exit code 13. Error file: prelaunch.err
body: <p>I am trying to execute a hello world like program in pyspark. I have created an EMR cluster thru boto3 and have added the step to execute my code.</p>
  <p>Step is :</p>
  <pre><code>            'Name': 'Run Step',
              'ActionOnFailure': 'CONTINUE',
              'HadoopJarStep': {
                  'Args': [
                      'spark-submit',
                      '--master', 'yarn',
                      '--deploy-mode', 'cluster',
                    #  '--py-files',
                      's3://bucket/s3csvload.py'
                  ],
                  'Jar': 'command-runner.jar'
              }
          }
  </code></pre>
  <p>The code I am trying to execute is</p>
  <pre><code>rom pyspark.sql import *
  from pyspark.sql.types import *
  from pyspark.sql.functions import *
  from pyspark.sql import SparkSession
  
  spark = SparkSession.builder.master('yarn').appName('DIF1').getOrCreate()
  
  Input_schema1 = StructType([StructField(&quot;sepal_length&quot;, DecimalType(), True),
                              StructField(&quot;sepal_width&quot;, DecimalType(), True),
                              StructField(&quot;petal_length&quot;, DecimalType(), True),
                              StructField(&quot;petal_width&quot;, DecimalType(), True),
                              StructField(&quot;species&quot;, StringType(), True)])
  
  lookup_df = spark.read \
                .option(&quot;header&quot;, &quot;true&quot;) \
                .option(&quot;inferSchema&quot;, &quot;true&quot;) \
                .option(&quot;schema&quot;, Input_schema1) \
                .csv(&quot;s3://bucket/iris.csv&quot;)
  
  lookup_df.write.csv(&quot;s3://bucket/Target&quot;)
  </code></pre>
  <p>The error I am facing is:</p>
  <pre><code>20/12/22 15:27:29 INFO Client: 
       client token: N/A
       diagnostics: Application application_1xxxx0_0003 failed 2 times due to AM Container for appattempt_16xxxxx10_0003_000002 exited with  exitCode: 13
  Failing this attempt.Diagnostics: [2020-12-22 15:27:28.643]Exception from container-launch.
  Container id: container_16xxxx10_0003_02_000001
  Exit code: 13
  
  [2020-12-22 15:27:28.644]Container exited with a non-zero exit code 13. Error file: prelaunch.err.
  Last 4096 bytes of prelaunch.err :
  Last 4096 bytes of stderr :
  </code></pre>
  <p>I have tried related links. Not much helpful.</p>
  <p>I guess I have to change something in spark session builder. But, not sure.
  Any helps appreciated. Thank you.</p>