title: How to apply udf on a dataframe and on a column in scala?
body: <p>I am beginner to scala. I tried <code>scala REPL</code> window in <code>intellij</code>.
  I have a sample df and trying to test <code>udf function</code> not <code>builtin</code> for understanding.</p>
  <p><em>df</em>:</p>
  <pre><code>scala&gt; import org.apache.spark.sql.SparkSession
   val spark: SparkSession = SparkSession.builder.appName(&quot;elephant&quot;).config(&quot;spark.master&quot;, &quot;local[*]&quot;).getOrCreate()
   val df = spark.createDataFrame(Seq((&quot;A&quot;,1),(&quot;B&quot;,2),(&quot;C&quot;,3))).toDF(&quot;Letter&quot;, &quot;Number&quot;)
   df.show()
  </code></pre>
  <p><em>output</em>:</p>
  <pre><code>|Letter|Number|
  +------+------+
  |     A|     1|
  |     B|     2|
  |     C|     3|
  +------+------+
  </code></pre>
  <p><em>udf for dataframe filter:</em></p>
  <pre><code>scala&gt; def kill_4(n: String) : Boolean = {
       | if (n ==&quot;A&quot;){ true} else {false}} // please validate if its correct ???
  </code></pre>
  <p><em>I tried</em></p>
  <pre><code>df.withColumn(&quot;new_col&quot;, kill_4(col(&quot;Letter&quot;))).show() // please tell correct way???
  </code></pre>
  <p><strong>error</strong>
  <code>error: type mismatch</code></p>
  <p>Second:
  I tried direct filter:</p>
  <pre><code>df.filter(kill_4(col(&quot;Letter&quot;))).show()
  </code></pre>
  <p><strong>output desired</strong></p>
  <pre><code>+------+------+
  |Letter|Number|
  +------+------+
  |     B|     2| 
  |     C|     3| 
  +------+------+-
  </code></pre>