title: How to use distributed Spark and Play Framework?
body: <p>How do you use Play Framework and a Spark cluster in development?</p>
  
  <p>I can run any Spark app with the master set to local[*]</p>
  
  <p>But if I set it to run on the cluster, I get this:</p>
  
  <pre><code>play.api.Application$$anon$1: Execution exception[[SparkException: Job aborted due to stage failure: Task 1 in stage 0.0 failed 4 times, most recent failure: Lost task 1.3 in stage 0.0 (TID 5, 192.168.1.239): java.lang.ClassNotFoundException: controllers.Application$$anonfun$test$1$$anonfun$2
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:348)
  at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:67)
  at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
  at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
  at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
  at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
  at org.apache.spark.scheduler.Task.run(Task.scala:86)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)
  </code></pre>
  
  <p>I understand the problem is that the distributed workers don't have my app classes loaded. </p>
  
  <p>So how do you use Spark under Lightbend Activator? It doesn't make any sense to submit a Play Framework app via the command line, it's supposed to run under Play so you can see the results in the browser. </p>
  
  <p>I downloaded the Lightbend sample Spark applications and they use local[*] for the Spark Master. If I switch to the spark://master:port url they all crash with the same problem.</p>
  
  <p>Does anyone know how to fix this? Thanks in advance.</p>