title: Timeout a pyspark job
body: <p>TL;DR</p>
  
  <p>Is there a way to timeout a pyspark job? I want a spark job running in cluster mode to be killed automatically if it runs longer than a pre-specified time.</p>
  
  <p>Longer Version:</p>
  
  <p>The cryptic timeouts listed in <a href="http://spark.apache.org/docs/latest/configuration.html" rel="nofollow noreferrer">the documentation</a> are at most 120s, except one which is infinity, but this one is only used if spark.dynamicAllocation.enabled   is set to true, but by default (I havent touch any config parameters on this cluster) it is false.</p>
  
  <p>I want to know because I have a code that for a particular pathological input will run extremely slow. For expected input the job will terminate in under an hour.Detecting the pathological input is as hard as trying to solve the problem, so I don't have the option of doing clever preprocessing. The details of the code are boring and irrelevant, so I'm going to spare you having to read them =)</p>
  
  <p>Im using pyspark so I was going to decorate the function causing the hang up like <a href="https://stackoverflow.com/questions/2281850/timeout-function-if-it-takes-too-long-to-finish">this</a> but it seems that this solution doesnt work in cluster mode. I call my spark code via spark-submit from a bash script, but so far as I know bash "goes to sleep" while the spark job is running and only gets control back once the spark job terminates, so I don't think this is an option.</p>
  
  <p>Actually, the bash thing might be a solution if I did something clever but I'd have to get the driver id for the job like <a href="https://stackoverflow.com/questions/30093959/stopping-a-running-spark-application">this</a>, and by now I'm thinking "this is too much thought and typing for something so simple as a timeout which ought to be built in."</p>