title: Amazon Elastic MapReduce - SIGTERM
body: <p>I have an EMR streaming job (Python) which normally works fine (e.g. 10 machines processing 200 inputs). However, when I run it against large data sets (12 machines processing a total of 6000 inputs, at about 20 seconds per input), after 2.5 hours of crunching I get the following error:</p>
  
  <pre><code>java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 143
  at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:372)
  at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:586)
  at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
  at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:441)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:377)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:396)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
  at org.apache.hadoop.mapred.Child.main(Child.java:249)
  </code></pre>
  
  <p>If I am reading this correctly, the subprocess failed with code 143 because someone sent a SIGTERM signal to the streaming job.</p>
  
  <p>Is my understanding correct? If so: When would the EMR infrastructure send a SIGTERM?</p>