title: Debug Apache Slider package?
body: <p>I went through the Slider Memcached Tutorial and was able to package/deploy/start the memcached container successfully; however when I package up a custom application, basically a Java jar plus dependencies, the container never launches succssfully.</p>
  
  <p>The application page show the app is in a FINISHED/FAILED state with this diagnostic:
  <a href="http://quickstart.cloudera:8088/cluster/app/application_1439926335194_0001" rel="nofollow">http://quickstart.cloudera:8088/cluster/app/application_1439926335194_0001</a></p>
  
  <p>Diagnostics: Unstable Application Instance : - failed with component MYAPP failed 'recently' 6 times (4 in startup); threshold is 5 - last failure: Failure container_1439926335194_0001_01_000008 on host quickstart.cloudera (0): <a href="http://quickstart.cloudera:19888/jobhistory/logs//quickstart.cloudera:8041/container_1439926335194_0001_01_000008/ctx/MYUSER" rel="nofollow">http://quickstart.cloudera:19888/jobhistory/logs//quickstart.cloudera:8041/container_1439926335194_0001_01_000008/ctx/MYUSER</a></p>
  
  <p>Part of the challenge in diagnosing the issue with the container is that the logs disappear after the application completes.
  <a href="http://quickstart.cloudera:8042/node/containerlogs/container_1439926335194_0001_01_000001/MYUSER" rel="nofollow">http://quickstart.cloudera:8042/node/containerlogs/container_1439926335194_0001_01_000001/MYUSER</a></p>
  
  <p>There is a troubleshooting page for slider which indicates that you can persist the logs beyond application completion:
  <a href="http://slider.incubator.apache.org/docs/troubleshooting.html" rel="nofollow">http://slider.incubator.apache.org/docs/troubleshooting.html</a></p>
  
  <p>Configuring YARN for better debugging
  One configuration to aid debugging is tell the nodemanagers to keep data for a short period after containers finish</p>
  
  <pre><code>&lt;!-- 10 minutes after a failure to see what is left in the directory--&gt;
  &lt;property&gt;
    &lt;name&gt;yarn.nodemanager.delete.debug-delay-sec&lt;/name&gt;
    &lt;value&gt;600&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>And I found this setting in Yarn - Configuration - NodeManager Base Group - Advanced - Localized Dir Delection Delay and changed it from the default of 0 to 1200; however even after I deploy client config, and restart Nodemanager + Yarn, even restart the VM, the logs are still getting deleted on container completion.</p>
  
  <p>I'm working on the CDH 5.3.0 Vitrualbox VM image and the cluster + services appear to be working normally as I start up the package.</p>
  
  <p><strong><em>EDIT:</em></strong></p>
  
  <p>Only error in the log I see is this:</p>
  
  <p>Role instance RoleInstance failed </p>
  
  <p>2015-08-19 10:59:21,819 [AMRM Callback Handler Thread] ERROR appmaster.SliderAppMaster - Role instance RoleInstance{role='SIMHASH', id='container_1439926335194_0002_01_000003', container=ContainerID=container_1439926335194_0002_01_000003 nodeID=quickstart.cloudera:8041 http=quickstart.cloudera:8042 priority=1073741825 resource=, createTime=1440007115649, startTime=1440007115674, released=false, roleId=1, host=quickstart.cloudera, hostURL=<a href="http://quickstart.cloudera:8042" rel="nofollow">http://quickstart.cloudera:8042</a>, state=5, placement=null, exitCode=0, command='python ./infra/agent/slider-agent/agent/main.py --label container_1439926335194_0002_01_000003___SIMHASH --zk-quorum localhost:2181 --zk-reg-path /registry/users/c4/services/org-apache-slider/simhash1 > /slider-agent.out 2>&amp;1 ; ', diagnostics='', output=null, environment=[LANGUAGE="en_US.UTF-8", AGENT_WORK_ROOT="$PWD", HADOOP_USER_NAME="C4", AGENT_LOG_ROOT="", PYTHONPATH="./infra/agent/slider-agent/", LC_ALL="en_US.UTF-8", SLIDER_PASSPHRASE="8R9ZPw3aZ20GFydi3OqvEtwYhh1qzfQBmWv6BjXepg3PCcyS8m", LANG="en_US.UTF-8"]} failed</p>