title: How to make sure my DataFrame frees its memory?
body: <p>I have a Spark/Scala job in which I do this:</p>
  
  <ul>
  <li>1: Compute a big DataFrame <code>df1</code> + <code>cache</code> it into memory</li>
  <li>2: Use <code>df1</code> to compute <code>dfA</code></li>
  <li>3: Read raw data into <code>df2</code> (again, its big) + <code>cache</code> it</li>
  </ul>
  
  <p>When performing (3), I do no longer need <code>df1</code>. I want to make sure its space gets freed. I <code>cached</code> at (1) because this DataFrame gets used in (2) and its the only way to make sure I do not recompute it each time but only once.</p>
  
  <p>I need to free its space and make sure it gets freed. What are my options?</p>
  
  <p>I thought of these, but it doesn't seem to be sufficient:</p>
  
  <ul>
  <li><code>df=null</code></li>
  <li><code>df.unpersist()</code></li>
  </ul>
  
  <p>Can you document your answer with a proper Spark documentation link?</p>