title: update multiple columns based on two columns in pyspark data frames
body: <p>I have a data frame like below in <code>pyspark</code>.</p>
  
  <pre><code>+--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  |     serial_number  |     rest_id  |     value  |     body  |     legs  |     face  |     idle  |
  +--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  | sn11               | rs1          | N          | Y         | N         | N         | acde      |
  | sn1                | rs1          | N          | Y         | N         | N         | den       |
  | sn1                | null         | Y          | N         | Y         | N         | can       |
  | sn2                | rs2          | Y          | Y         | N         | N         | aeg       |
  | null               | rs2          | N          | Y         | N         | Y         | ueg       |
  +--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  </code></pre>
  
  <p>Now I want to <code>update</code> some of the column while checking some column values.</p>
  
  <p>I want to update the <code>value</code> when the any given <code>serial_number</code> or <code>rest_id</code> has value  <code>Y</code> then all <code>values</code> of that particular <code>serial_number</code> or <code>rest_id</code> should be updated to Y. if not then what ever values they have.</p>
  
  <p>I have done like below.</p>
  
  <pre><code>df.alias('a').join(df.filter(col('value')='Y').alias('b'),on=(col('a.serial_number') == col('b.serial_number')) | (col('a.rest_id') == col('b.rest_id')), how='left').withColumn('final_value',when(col('b.value').isNull(), col('a.value')).otherwise(col('b.value'))).select('a.serial_number','a.rest_id','a.body', 'a.legs', 'a.face', 'a.idle', 'final_val')
  </code></pre>
  
  <p>I got the result I want.</p>
  
  <p>Now I want to repeat the same for columns <code>body</code>, <code>legs</code> and <code>face</code> as well.</p>
  
  <p>I can do like above for all columns <code>individually</code>, I mean to say <code>3</code> join statements. But I want to update all the <code>4</code> columns in a single statement.</p>
  
  <p>How can I do that?</p>
  
  <p><code>Expected result</code></p>
  
  <pre><code>+--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  |     serial_number  |     rest_id  |     value  |     body  |     legs  |     face  |     idle  |
  +--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  | sn11               | rs1          | N          | Y         | N         | N         | acde      |
  | sn1                | rs1          | Y          | Y         | Y         | N         | den       |
  | sn1                | null         | Y          | Y         | Y         | N         | can       |
  | sn2                | rs2          | Y          | Y         | N         | Y         | aeg       |
  | null               | rs2          | Y          | Y         | N         | Y         | ueg       |
  +--------------------+--------------+------------+-----------+-----------+-----------+-----------+
  </code></pre>