title: Deploying application with spark-submit: Application is added to the scheduler and is not yet activated
body: <p>I have VirtualBox with Linux Centos 12G memory. I need to deploy 2 applications to the hadoop running in non-distributed configuration. This is my YARN config:</p>
  
  <pre><code>&lt;configuration&gt;
  &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.pmem-check-enabled&lt;/name&gt;
      &lt;value&gt;false&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
     &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.resourcemanager.address&lt;/name&gt;
     &lt;value&gt;0.0.0.0:8032&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;yarn.scheduler.maximum-allocation-vcores&lt;/name&gt;
    &lt;value&gt;130&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.nodemanager.vmem-check-enabled&lt;/name&gt;
     &lt;value&gt;false&lt;/value&gt;
     &lt;description&gt;Whether virtual memory limits will be enforced for containers&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.scheduler.maximum-allocation-mb&lt;/name&gt;
     &lt;value&gt;4048&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.nodemanager.vmem-pmem-ratio&lt;/name&gt;
     &lt;value&gt;1&lt;/value&gt;
     &lt;description&gt;Ratio between virtual memory to physical memory when
  setting memory limits for containers&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt;
     &lt;name&gt;yarn.scheduler.capacity.maximum-am-resource-percent&lt;/name&gt;
     &lt;value&gt;1&lt;/value&gt;
  &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>I deploy first application and it runs correctly:</p>
  
  <blockquote>
    <p>spark-submit --master yarn --deploy-mode client --name
    OryxBatchLayer-ALSExample --class com.cloudera.oryx.batch.Main 
    --files oryx.conf --driver-memory 500m --driver-java-options "-Dconfig.file=oryx.conf" --executor-memory 500m --executor-cores 1
    --conf spark.executor.extraJavaOptions="-Dconfig.file=oryx.conf" --conf spark.ui.port=4040 --conf spark.io.compression.codec=lzf --conf spark.logConf=true --conf
    spark.serializer=org.apache.spark.serializer.KryoSerializer --conf
    spark.speculation=true --conf spark.ui.showConsoleProgress=false
    --conf spark.dynamicAllocation.enabled=false --num-executors=1 oryx-batch-2.8.0-SNAPSHOT.jar</p>
  </blockquote>
  
  <p>YARN manager at 8088 indicates that I'm using 2 of 8 vcores and 2 of 8g memory:</p>
  
  <p><a href="https://i.stack.imgur.com/3704c.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/3704c.png" alt="enter image description here"></a></p>
  
  <p>Now I deploy my second application:</p>
  
  <blockquote>
    <p>spark-submit --master yarn --deploy-mode client --name
    OryxSpeedLayer-ALSExample --class com.cloudera.oryx.speed.Main --files
    oryx.conf --driver-memory 500m --driver-java-options
    "-Dconfig.file=oryx.conf" --executor-memory 500m --executor-cores 1
    --conf spark.executor.extraJavaOptions="-Dconfig.file=oryx.conf" --conf spark.ui.port=4041 --conf spark.io.compression.codec=lzf --conf spark.logConf=true --conf
    spark.serializer=org.apache.spark.serializer.KryoSerializer --conf
    spark.speculation=true --conf spark.ui.showConsoleProgress=false
    --conf spark.dynamicAllocation.enabled=false --num-executors=1 oryx-speed-2.8.0-SNAPSHOT.jar</p>
  </blockquote>
  
  <p>but this time I get a warning, also it seems that second application is frozen, at least it doesn't allocate memory:</p>
  
  <blockquote>
    <p>2018-08-06 04:49:10 INFO  Client:54 -
             client token: N/A
             diagnostics: [Mon Aug 06 04:49:09 -0400 2018] Application is added to the scheduler and is not yet activated. Queue's AM resource
    limit exceeded.  Details : AM Partition = ; AM
    Resource Request = ; Queue Resource Limit for
    AM = ; User AM Resource Limit of the queue =
    ; Queue AM Resource Usage = ;
             ApplicationMaster host: N/A
             ApplicationMaster RPC port: -1
             queue: default
             start time: 1533545349902
             final status: UNDEFINED
             tracking URL: <a href="http://master:8088/proxy/application_1533542648791_0002/" rel="nofollow noreferrer">http://master:8088/proxy/application_1533542648791_0002/</a>
             user: osboxes</p>
  </blockquote>
  
  <p><a href="https://i.stack.imgur.com/bT77b.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/bT77b.png" alt="enter image description here"></a></p>
  
  <p>What is the rootcause of the problem? How can I increase <strong>Queue Resource Limit for AM</strong> and <strong>User AM Resource Limit of the queue</strong>?</p>