title: Oozie SparkAction failing
body: <p>On HDP 2.3.4 I'm trying to test out Oozie's SparkAction.</p>
  
  <p>The Spark (1.5.2) Application that I wrote is trivial and solely for testing out Oozie (4.2.0):</p>
  
  <pre><code>val tbl = sqlContext.sql("SELECT * FROM tbl")
  val count = tbl.count   
  log.info(s"The table has ${count} records.")
  </code></pre>
  
  <p>This application works when using <code>spark-submit</code>, both in YARN-Client and YARN-Cluster modes.</p>
  
  <p>My <code>job.properties</code> and <code>workflow.xml</code> files are as follows:</p>
  
  <p><strong>job.properties</strong></p>
  
  <pre><code>nameNode=hdfs://myhost.com:8020
  jobTracker=myhost.com:8032
  queueName=default
  projectRoot=user/${user.name}/workflows/sparkaction-test
  
  master=yarn-cluster
  mode=cluster
  class=com.myCompany.SparkActionTest
  hiveSite=hive-site.xml
  jars=${nameNode}/${projectRoot}/lib/sparkaction-test_2.10-1.0.jar
  
  
  oozie.use.system.libpath=true
  oozie.wf.application.path=${nameNode}/${projectRoot}
  spark.yarn.historyServer.address=http://myhost.com:18080/
  spark.eventLog.dir=${nameNode}/user/spark/applicationHistory
  spark.eventLog.enabled=true
  </code></pre>
  
  <p><strong>workflow.xml</strong></p>
  
  <pre><code>&lt;workflow-app name="spark-test-wf" xmlns="uri:oozie:workflow:0.4"&gt;
      &lt;start to="spark-test"/&gt;
      &lt;action name="spark-test"&gt;
          &lt;spark xmlns="uri:oozie:spark-action:0.1"&gt;
              &lt;job-tracker&gt;${jobTracker}&lt;/job-tracker&gt;
              &lt;name-node&gt;${nameNode}&lt;/name-node&gt;
              &lt;configuration&gt;
                  &lt;property&gt;
                      &lt;name&gt;mapred.compress.map.output&lt;/name&gt;
                      &lt;value&gt;true&lt;/value&gt;
                  &lt;/property&gt;
              &lt;/configuration&gt;
              &lt;master&gt;${master}&lt;/master&gt;
              &lt;mode&gt;${mode}&lt;/mode&gt;
              &lt;name&gt;Testing Spark Action&lt;/name&gt;
              &lt;class&gt;${class}&lt;/class&gt;
          &lt;jar&gt;${jars}&lt;/jar&gt;
              &lt;spark-opts&gt;--files ${hiveSite}&lt;/spark-opts&gt;
           &lt;/spark&gt;
          &lt;ok to="end"/&gt;
          &lt;error to="errorcleanup" /&gt;
      &lt;/action&gt;
  
      &lt;kill name="errorcleanup"&gt;
        &lt;message&gt;Spark Test WF failed. [${wf:errorMessage(wf:lastErrorNode())}]&lt;/message&gt;
      &lt;/kill&gt;
      &lt;end name ="end"/&gt;
  &lt;/workflow-app&gt;
  </code></pre>
  
  <p>I cleaned out the Oozie sharelib for Spark (moved them into a different directory) and left only the following jars in them:</p>
  
  <pre><code>datanucleus-api-jdo-3.2.6.jar
  datanucleus-core-3.2.10.jar
  datanucleus-rdbms-3.2.9.jar
  oozie-sharelib-spark-4.2.0.2.3.4.0-3485.jar
  spark-1.5.2.2.3.4.0-3485-yarn-shuffle.jar
  spark-assembly-1.5.2.2.3.4.0-3485-hadoop2.7.1.2.3.4.0-3485.jar
  spark-examples-1.5.2.2.3.4.0-3485-hadoop2.7.1.2.3.4.0-3485.jar
  </code></pre>
  
  <p>Unfortunately, the Oozie workflow fails with the following error message:</p>
  
  <p>The workflow is killed after being in running status for about 20 minutes and the error message is as follows: <strong>Call From host.xxx.com/x.x.x.x to 0.0.0.0:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: <a href="http://wiki.apache.org/hadoop/ConnectionRefused" rel="nofollow">http://wiki.apache.org/hadoop/ConnectionRefused</a></strong></p>
  
  <p>I can ping 0.0.0.0 but can't telnet port 8032.</p>