title: Is it possible to store a numpy array in a Spark Dataframe Column?
body: <p>I have a <code>dataframe</code> and I apply a function to it. This function returns an <code>numpy array</code> the code looks like this:</p>
  
  <pre><code>create_vector_udf = udf(create_vector, ArrayType(FloatType()))
  dataframe = dataframe.withColumn('vector', create_vector_udf('text'))
  dmoz_spark_df.select('lang','url','vector').show(20)
  </code></pre>
  
  <p>Now spark seems not to be happy with this and does not accept <code>ArrayType(FloatType())</code>
  I get the following error message:
  <code>net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct)</code></p>
  
  <p>I could just <code>numpyarray.tolist()</code> and return a list version of it, but obviously I would always have to recreate the <code>array</code> if I want to use it with <code>numpy</code>. </p>
  
  <p>so is there a way to store a <code>numpy array</code>  in a <code>dataframe</code> <code>column</code>?</p>