title: How to append an element to an array column of a Spark Dataframe?
body: <p>Suppose I have the following DataFrame:</p>
  
  <pre><code>scala&gt; val df1 = Seq("a", "b").toDF("id").withColumn("nums", array(lit(1)))
  df1: org.apache.spark.sql.DataFrame = [id: string, nums: array&lt;int&gt;]
  
  scala&gt; df1.show()
  +---+----+
  | id|nums|
  +---+----+
  |  a| [1]|
  |  b| [1]|
  +---+----+
  </code></pre>
  
  <p>And I want to add elements to the array in the <code>nums</code> column, so that I get something like the following:</p>
  
  <pre><code>+---+-------+
  | id|nums   |
  +---+-------+
  |  a| [1,5] |
  |  b| [1,5] |
  +---+-------+
  </code></pre>
  
  <p>Is there a way to do this using the <code>.withColumn()</code> method of the DataFrame? E.g. </p>
  
  <pre><code>val df2 = df1.withColumn("nums", append(col("nums"), lit(5))) 
  </code></pre>
  
  <p>I've looked through the API documentation for Spark, but can't find anything that would allow me to do this. I could probably use <code>split</code> and <code>concat_ws</code> to hack something together, but I would prefer a more elegant solution if one is possible. Thanks.</p>