title: How to configure Hive to use Spark execution engine on Google Dataproc?
body: <p>I'm trying to configure Hive, running on Google Dataproc image v1.1 (so Hive 2.1.0 and Spark 2.0.2), to use Spark as an execution engine instead of the default MapReduce one.</p>
  
  <p>Following the instructions here <a href="https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started" rel="nofollow noreferrer">https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started</a> doesn't really help, I keep getting <code>Error running query: java.lang.NoClassDefFoundError: scala/collection/Iterable</code> errors when I set <code>hive.execution.engine=spark</code>.</p>
  
  <p>Does anyone know the specific steps to get this running on Dataproc? From what I can tell it should just be a question of making Hive see the right JARs, since both Hive and Spark are already installed and configured on the cluster, and using Hive from Spark (so the other way around) works fine.</p>