title: Pyspark Multiple JOINS Column <> Row values: Reducing Actions
body: <p>I have a master table 'Table 1' with 3 columns(Shown below). Tables 2.1, 3.1 &amp; 4.1 are for 3 unique dates present in Table 1 and need to be populated in column 'Points 1'. Similarly, Tables 2.2, 3.2 &amp; 4.2 are for same 3 unique dates present in Table 1 and need to be populated in column 'Points 2'.</p>
  <p><a href="https://i.stack.imgur.com/c97jk.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/c97jk.png" alt="enter image description here" /></a></p>
  <p><strong>Current Approach:</strong></p>
  <pre><code>df1 = spark.table(&quot;Table1&quot;)
  df2_1 = spark.table(&quot;table2.1&quot;)
  df2_1 = withColumn(&quot;Date&quot;, lit(3312019))
  
  
  df3 = df1.join(df2_1, df1.ID==df2.1==ID &amp; df1.Date==df2_1.Date, 'left')
  df4 = df3.withColumn('Points', when(df3.Category==A, col('A'))
                                .when(df3.Category==B, col('B'))
                                .when(df3.Category==C, col('C'))
                                .when(df3.Category==D, col('D'))
                                .otherwise(lit(None)))
  
  </code></pre>
  <p>Current Approach makes my code lengthy if implemented for all 6 tables, any suggestions to shorten it and reduce multiple actions?</p>