title: pyspark: merge (outer-join) two data frames
body: <p>I have the following two data frames:</p>
  
  <pre><code>DF1:
  
      Id | field_A | field_B | field_C | field_D
       1 |   cat   |  12     |   black | 11
       2 |   dog   | 128     |   white | 19
       3 |   dog   |  35     |  yellow | 20
       4 |   dog   |  21     |   brown |  4
       5 |  bird   |  10     |    blue |  7
       6 |   cow   |  99     |   brown | 34
  </code></pre>
  
  <p>and</p>
  
  <pre><code>DF2:
  
      Id | field_B | field_C | field_D | field_E
       3 |  35     |  yellow | 20      |   123   
       5 |  10     |    blue |  7      |   454   
       6 |  99     |   brown | 34      |   398   
  </code></pre>
  
  <p>And I am hoping to get the new_DF as</p>
  
  <pre><code>    Id | field_A | field_B | field_C | field_D | field_E
       1 |   cat   |  12     |   black | 11      |
       2 |   dog   | 128     |   white | 19      |
       3 |   dog   |  35     |  yellow | 20      |  123
       4 |   dog   |  21     |   brown |  4      |  
       5 |  bird   |  10     |    blue |  7      |  454
       6 |   cow   |  99     |   brown | 34      |  398
  </code></pre>
  
  <hr>
  
  <p>Could this be achieved by data frame operations? Thanks!</p>