title: Aggreagation and differences between two data frames - Pyspark
body: <p>My data frame looks like below</p>
  
  <p>DF1 :</p>
  
  <pre><code>A    B    Value
  0   abc   200
  xyz  0     300
  </code></pre>
  
  <p>DF2:</p>
  
  <pre><code>A    B    Value
  abc   0    400
  0     xyz    500
  </code></pre>
  
  <p>I need to calculate the difference between the two dataframes for Value(column)
  I am using "coalesce" function to get the IDs</p>
  
  <p>ID    Value
  abc   200
  xyz    300</p>
  
  <p>similarly for DF2</p>
  
  <p>Need help in :
  1) changing/adding a column name which could contain results from coalesce function
  Tried below code: doesnt work, it gives me null values</p>
  
  <pre><code>df1.select(coalesce(df1["A"],df1["B"]).alias("ID"))
  </code></pre>
  
  <p>Also I want to display as 
  ID , Value
  abc  200</p>
  
  <p>2) How do I join these two dataframes based on ID and then subtract values in the same query and display both ID as ID and difference as new column called as "diff"</p>
  
  <pre><code>Expected OUtput:
  
  ID    Value
  abc   200
  xyz   200
  </code></pre>