title: Spark streaming with python: how to add a UUID column?
body: <p>I would like to add a column with a generated id to my data frame. I have tried:</p>
  
  <pre><code>uuidUdf = udf(lambda x: str(uuid.uuid4()), StringType())
  df = df.withColumn("id", uuidUdf())
  </code></pre>
  
  <p>however, when I do this, nothing is written to my output directory. When I remove these lines, everything works fine so there must be some error but I don't see anything in the console.</p>
  
  <p>I have tried using monotonically_increasing_id() instead of generating a UUID but in my testing, this produces many duplicates. I need a unique identifier (does not have to be a UUID specifically). </p>
  
  <p>How can I do this?</p>