title: How to convert a DataFrame back to normal RDD in pyspark?
body: <p>I need to use the </p>
  
  <pre><code>(rdd.)partitionBy(npartitions, custom_partitioner)
  </code></pre>
  
  <p>method that is not available on the DataFrame.  All of the DataFrame methods refer only to DataFrame results.  So then how to create an RDD from the DataFrame data?</p>
  
  <p>Note: this is a change (in 1.3.0) from 1.2.0.</p>
  
  <p><strong>Update</strong> from the answer from @dpangmao: the method is .rdd. I was interested to understand if (a) it were public and (b) what are the performance implications.</p>
  
  <p>Well (a) is yes and (b) - well you can see here that there are significant perf implications: a new RDD must be created  by invoking <strong>mapPartitions</strong> :</p>
  
  <p>In <strong>dataframe.py</strong> (note the file name changed as well (was sql.py):</p>
  
  <pre><code>@property
  def rdd(self):
      """
      Return the content of the :class:`DataFrame` as an :class:`RDD`
      of :class:`Row` s.
      """
      if not hasattr(self, '_lazy_rdd'):
          jrdd = self._jdf.javaToPython()
          rdd = RDD(jrdd, self.sql_ctx._sc, BatchedSerializer(PickleSerializer()))
          schema = self.schema
  
          def applySchema(it):
              cls = _create_cls(schema)
              return itertools.imap(cls, it)
  
          self._lazy_rdd = rdd.mapPartitions(applySchema)
  
      return self._lazy_rdd
  </code></pre>