title: Failed to report status for 600 seconds. Killing! Reporting progress in hadoop
body: <p>I receive the following error: </p>
  
  <pre><code>Task attempt_201304161625_0028_m_000000_0 failed to report status for 600 seconds. Killing! 
  </code></pre>
  
  <p>for my Map jobs. This question is similar to <a href="https://stackoverflow.com/questions/5864589/how-to-fix-task-attempt-201104251139-0295-r-000006-0-failed-to-report-status-fo">this</a>, <a href="https://stackoverflow.com/questions/15281307/the-reduce-fails-due-to-task-attempt-failed-to-report-status-for-600-seconds-ki">this</a>, and <a href="https://stackoverflow.com/questions/14496053/how-to-fake-task-reporting-in-hadoop-job">this</a>. However, <strong>I do not want to increase the default time before hadoop kills a task that doesn't report progress</strong>, i.e.,</p>
  
  <pre><code>Configuration conf=new Configuration();
  long milliSeconds = 1000*60*60;
  conf.setLong("mapred.task.timeout", milliSeconds);
  </code></pre>
  
  <p>Instead, I want to periodically report progress using either <code>context.progress()</code>, <code>context.setStatus("Some Message")</code> or <code>context.getCounter(SOME_ENUM.PROGRESS).increment(1)</code> or something similar. However, this still causes the job to be killed. Here are the snippets of code where I am attempting to report progress. The mapper:</p>
  
  <pre><code>protected void map(Key key, Value value, Context context) throws IOException, InterruptedException {
  
      //do some things
      Optimiser optimiser = new Optimiser();
      optimiser.optimiseFurther(&lt;some parameters&gt;, context);
      //more things
      context.write(newKey, newValue);
  }
  </code></pre>
  
  <p>the optimiseFurther method within the Optimiser class:</p>
  
  <pre><code>public void optimiseFurther(&lt;Some parameters&gt;, TaskAttemptContext context) {
  
      int count = 0;
      while(something is true) {
          //optimise
  
          //try to report progress
          context.setStatus("Progressing:" + count);
          System.out.println("Optimise Progress:" + context.getStatus());
          context.progress();
          count++;
      }
  }
  </code></pre>
  
  <p>The output from a mapper shows the status is being updated:</p>
  
  <pre><code>Optimise Progress:Progressing:0
  Optimise Progress:Progressing:1
  Optimise Progress:Progressing:2
  ...
  </code></pre>
  
  <p>However, the job is still being killed after the default amount of time. Am I using the context in the wrong way? Is there anything else I need to do in the job setup in order to report the progress successfully? </p>