title: spark create Dataframe in UDF
body: <p>I have a example, want to create Dataframe in a UDF. Something like the one below</p>
  
  <pre><code>import org.apache.spark.ml.classification.LogisticRegressionModel
  import org.apache.spark.ml.linalg.Vector
  import org.apache.spark.ml.feature.VectorAssembler
  </code></pre>
  
  <p>data to Dataframe</p>
  
  <pre><code>    val df = Seq((1,1,34,23,34,56),(2,1,56,34,56,23),(3,0,34,23,23,78),(4,0,23,34,78,23),(5,1,56,23,23,12),
  (6,1,67,34,56,34),(7,0,23,23,23,56),(8,0,12,34,45,89),(9,1,12,34,12,34),(10,0,12,34,23,34)).toDF("id","label","tag1","tag2","tag3","tag4")
      val assemblerDF = new VectorAssembler().setInputCols(Array("tag1", "tag2", "tag3","tag4")).setOutputCol("features")
      val data = assemblerDF.transform(df)
      val Array(train,test) = data.randomSplit(Array(0.6, 0.4), seed = 11L)
      val testData=test.toDF    
  
      val loadmodel=LogisticRegressionModel.load("/user/xu/savemodel")
      sc.broadcast(loadmodel)
      val assemblerFe = new VectorAssembler().setInputCols(Array("a", "b", "c","d")).setOutputCol("features")
      sc.broadcast(assemblerFe)
  </code></pre>
  
  <p>UDF</p>
  
  <pre><code>    def predict(predictSet:Vector):Double={
          val set=Seq((1,2,3,4)).toDF("a","b","c","d")
          val predata = assemblerFe.transform(set)
          val result=loadmodel.transform(predata)
          result.rdd.take(1)(0)(3).toString.toDouble}
  
      spark.udf.register("predict", predict _)
      testData.registerTempTable("datatable")
      spark.sql("SELECT predict(features) FROM datatable").take(1)
  </code></pre>
  
  <p>i get an error like</p>
  
  <pre><code>ERROR Executor: Exception in task 3.0 in stage 4.0 (TID 7) [Executor task launch worker for task 7]
  org.apache.spark.SparkException: Failed to execute user defined function($anonfun$1: (vector) =&gt; double)
  </code></pre>
  
  <p>and</p>
  
  <pre><code>WARN TaskSetManager: Lost task 3.0 in stage 4.0 (TID 7, localhost, executor driver): org.apache.spark.SparkException: Failed to execute user defined function($anonfun$1: (vector) =&gt; double)
  </code></pre>
  
  <p>Is dataframe not supported? I am using Spark 2.3.0 and Scala 2.11. thanks</p>