title: Spark Submit Issue
body: <p>I am trying to run a fat jar on a Spark cluster using Spark submit.
  I made the cluster using "spark-ec2" executable in Spark bundle on AWS.</p>
  
  <p>The command I am using to run the jar file is</p>
  
  <pre><code>bin/spark-submit --class edu.gatech.cse8803.main.Main --master yarn-cluster ../src1/big-data-hw2-assembly-1.0.jar
  </code></pre>
  
  <p>In the beginning it was giving me the error that at least one of the <strong>HADOOP_CONF_DIR</strong> or <strong>YARN_CONF_DIR</strong> environment variable must be set.
  I didn't know what to set them to, so I used the following command</p>
  
  <pre><code>export HADOOP_CONF_DIR=/mapreduce/conf
  </code></pre>
  
  <p>Now the error has changed to</p>
  
  <pre><code>Could not load YARN classes. This copy of Spark may not have been compiled with YARN support.
  Run with --help for usage help or --verbose for debug output
  </code></pre>
  
  <p>The home directory structure is as follows</p>
  
  <pre><code>ephemeral-hdfs  hadoop-native  mapreduce  persistent-hdfs  scala  spark  spark-ec2  src1  tachyon
  </code></pre>
  
  <p>I even set the YARN_CONF_DIR variable to the same value as HADOOP_CONF_DIR, but the error message is not changing. I am unable to find any documentation that highlights this issue, most of them just mention these two variables and give no further details.</p>