title: Spark join not working due to an org.apache.spark.sql.AnalysisException: Cannot resolve column name
body: <p>I have case classes like these and I am trying to join the datasets created by the objects of the case classes.</p>
  <pre><code> case class RemoteCopyGroup(
      ts: Long,
      systemId: String,
      name: String,
      groupId: Int,
      role: String,
      mode: String,
      remoteGroupName: String)
  
      case class RemoteCopyGroupTarget(
       ts: Long,
       systemId: String,
       groupId: Int,
       targetName: String,
       groupName: String)
  
  
   val rcg1 = RemoteCopyGroup(
      ts = 1000,
      systemId = &quot;ABC&quot;,
      name = &quot;ABC&quot;,
      groupId = 100,
      role = &quot;ABC&quot;,
      mode = &quot;ABC&quot;,
      remoteGroupName = &quot;ABC&quot;)
  
    val rcg2 = RemoteCopyGroup(
      ts = 1000,
      systemId = &quot;ABC&quot;,
      name = &quot;ABC&quot;,
      groupId = 101,
      role = &quot;ABC&quot;,
      mode = &quot;ABC&quot;,
      remoteGroupName = &quot;ABC&quot;)
  
    val rcg3 = RemoteCopyGroup(
      ts = 1000,
      systemId = &quot;XYZ&quot;,
      name = &quot;ABC&quot;,
      groupId = 101,
      role = &quot;ABC&quot;,
      mode = &quot;ABC&quot;,
      remoteGroupName = &quot;ABC&quot;)
  
    val ds1 = List(rcg1, rcg2, rcg3).toDS()
  
     val rcgt1 = RemoteCopyGroupTarget(ts = 2000, systemId = &quot;ABC&quot;, groupId = 50, 
         targetName = &quot;ABC&quot;, groupName = &quot;XYZ&quot;)
     val rcgt2 = RemoteCopyGroupTarget(ts = 3000, systemId = &quot;DEF&quot;, groupId = 101, 
         targetName = &quot;ABC&quot;, groupName = &quot;XYZ&quot;)
  
      val ds2 = List(rcgt1, rcgt2).toDS()
  
      val joined = ds1.join(ds2, ds1.col(&quot;groupId&quot;) === ds2.col(&quot;groupId&quot;), &quot;inner&quot;)
  
      joined.show()
  </code></pre>
  <p>However I get an error</p>
  <pre><code>   Exception in thread &quot;main&quot; org.apache.spark.sql.AnalysisException: Cannot resolve 
     column name &quot;groupId&quot; among (value)
  </code></pre>
  <p>Although I have <code>groupId</code> present in the schema !! How to fix this ? Should I write something to make 'groupId' explicitly recognized ?</p>