title: PySpark Dynamic Join Condition
body: <p>I have list of PK columns. I am storing primary keys because the number of primary keys for each table might change.  I want to join two data frames on the basis of the columns in <code>pk_list</code>.</p>
  
  <pre><code>pk_list=['col1',col2', .... 'coln']
  </code></pre>
  
  <p>Right now my code looks like this:</p>
  
  <pre><code>full_load_tbl_nc = full_load_tbl.join(delta_load_tbl, (col(f) == col(s) for (f,s) in zip(pk_list,pk_list) ) , "leftanti")
  </code></pre>
  
  <p>When I run the code I get this error:</p>
  
  <blockquote>
    <p>casting list pyspark.sql.column also fails col(pk_list)
    with import from pyspark.sql.functions import col 
    File "/mnt/yarn/usercache/root/appcache/application_1544185829274_0001/container_1544185829274_0001_01_000001/pyspark.zip/pyspark/sql/dataframe.py", line 818, in join
    AssertionError: on should be Column or list of Column</p>
  </blockquote>