title: PySpark convert empty string to null and write in Parquet
body: <p>I am trying to convert empty strings to Null (None) and then write out in Parquet format.</p>
  
  <p>However, the output is still an empty string and not Null (None). Any ideas what I need to change? I am using <code>Spark 2.4</code> and <code>Python 3</code></p>
  
  <p><strong>Note:</strong> I am checking columns for String Data Type before applying the below, but I have omitted for simplicity of this question.</p>
  
  <pre><code>from pyspark.sql import functions as psf
  
  for column in df.columns:
      df_new = df.withColumn(
          column,
          psf.when(
              psf.length(
                  psf.trim(
                      psf.col(column)
                  )
              ) != 0,
              psf.trim(psf.col(column))).otherwise(None))
  </code></pre>