title: Java Virtual Machine error with Apache Spark after upgrade to 2.2.1
body: <p>I upgraded Apache Spark from 2.1.1 to 2.2.1 (without making any other changes), but now I can't run <code>spark-submit</code> or the pyspark shell without errors.</p>
  
  <p>If I try to run the pi calculation example from the command line with <code>./bin/run-example SparkPi 10</code> I get the following error:</p>
  
  <pre><code>Exception in thread "main" java.lang.VerifyError: 
  class org.apache.spark.sql.execution.LogicalRDD overrides final method sameResult.
  (Lorg/apache/spark/sql/catalyst/plans/QueryPlan;)Z
  </code></pre>
  
  <p>Alternatively, if I try to use the shell by running <code>./bin/pyspark</code> I get: </p>
  
  <pre><code>Exception in thread "Thread-2" java.lang.VerifyError: class org.apache.spark.sql.execution.LogicalRDD overrides final method sameResult.(Lorg/apache/spark/sql/catalyst/plans/QueryPlan;)Z
  
  ...
  
  ERROR:root:Exception while sending command.
  Traceback (most recent call last):
    File "/usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 883, in send_command
      response = connection.send_command(command)
    File "/usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1040, in send_command
      "Error while receiving", e, proto.ERROR_ON_RECEIVE)
  Py4JNetworkError: Error while receiving
  Exception in thread "Thread-16" java.lang.VerifyError: class org.apache.spark.sql.execution.LogicalRDD overrides final method sameResult.(Lorg/apache/spark/sql/catalyst/plans/QueryPlan;)Z
  
  ...
  
  ERROR:root:Exception while sending command.
  Traceback (most recent call last):
    File "/usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 883, in send_command
      response = connection.send_command(command)
    File "/usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1040, in send_command
      "Error while receiving", e, proto.ERROR_ON_RECEIVE)
  Py4JNetworkError: Error while receiving
  Traceback (most recent call last):
    File "/usr/local/spark/python/pyspark/shell.py", line 47, in &lt;module&gt;
      spark = SparkSession.builder.getOrCreate()
    File "/usr/local/spark/python/pyspark/sql/session.py", line 177, in getOrCreate
      session = SparkSession(sc)
    File "/usr/local/spark/python/pyspark/sql/session.py", line 211, in __init__
      jsparkSession = self._jvm.SparkSession(self._jsc.sc())
    File "/usr/local/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1532, in __getattr__
  py4j.protocol.Py4JError: SparkSession does not exist in the JVM
  </code></pre>
  
  <p>How do I fix this?</p>