title: Spark dataframe replace values of specific columns in a row with Nulls
body: <p>I am facing a problem when trying to replace the values of specific columns of a Spark dataframe with nulls.
  I have a dataframe with more than fifty columns of which two are key columns. I want to create a new dataframe with same schema and the new dataframe should have values from the key columns and null values in non-key columns. 
  I tried the following ways but facing issues:</p>
  
  <pre><code>//old_df is the existing Dataframe 
  val key_cols = List("id", "key_number")
  val non_key_cols = old_df.columns.toList.filterNot(key_cols.contains(_))
  
  val key_col_df = old_df.select(key_cols.head, key_cols.tail:_*)
  val non_key_cols_df = old_df.select(non_key_cols.head, non_key_cols.tail:_*)
  val list_cols = List.fill(non_key_cols_df.columns.size)("NULL")
  val rdd_list_cols = spark.sparkContext.parallelize(Seq(list_cols)).map(l =&gt; Row(l:_*))
  val list_df = spark.createDataFrame(rdd_list_cols, non_key_cols_df.schema)
  
  val new_df = key_col_df.crossJoin(list_df)
  </code></pre>
  
  <p>This approach was good when I only have string type columns in the <code>old_df</code>. But I have some columns of double type and int type which is throwing error because the rdd is a list of null strings. </p>
  
  <p>To avoid this I tried the <code>list_df</code> as an empty dataframe with schema as the <code>non_key_cols_df</code> but the result of <code>crossJoin</code> is an empty dataframe which I believe is because one dataframe is empty. </p>
  
  <p>My requirement is to have the <code>non_key_cols</code> as a single row dataframe with Nulls so that I can do <code>crossJoin</code> with <code>key_col_df</code> and form the required <code>new_df</code>. </p>
  
  <p>Also any other easier way to update all columns except key columns of a dataframe to nulls will resolve my issue. Thanks in advance</p>