title: Modifying and Building Spark core
body: <p>I am trying to make a modification to the Apache Spark source code. I created a new method and added it to the RDD.scala file within the Spark source code I downloaded. After making the modification to RDD.scala, I built Spark using</p>
  
  <pre><code>mvn -Dhadoop.version=2.2.0 -DskipTests clean package
  </code></pre>
  
  <p>I then created a sample Scala Spark Application as mentioned <a href="http://people.apache.org/~tdas/spark-1.2-temp/quick-start.html" rel="nofollow">here</a>
  I tried using the new function I created, and I got a compilation error when using sbt to create a jar for Spark. How exactly do I compile Spark with my modification and attach the modified jar to my project? The file I modified is RDD.scala within the core project. I run sbt package from the root dir of my Spark Application Project.</p>
  
  <p>Here is the sbt file:</p>
  
  <pre><code>name := "N Spark"
  
  version := "1.0"
  
  scalaVersion := "2.11.6"
  
  libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.3.0"
  </code></pre>
  
  <p>Here is the error:</p>
  
  <pre><code>sbt package
  [info] Loading global plugins from /Users/Raggy/.sbt/0.13/plugins
  [info] Set current project to Noah Spark (in build file:/Users/r/Downloads/spark-proj/n-spark/)
  [info] Updating {file:/Users/r/Downloads/spark-proj/n-spark/}n-spark...
  [info] Resolving jline#jline;2.12.1 ...
  [info] Done updating.
  [info] Compiling 1 Scala source to /Users/r/Downloads/spark-proj/n-spark/target/scala-2.11/classes...
  [error] /Users/r/Downloads/spark-proj/n-spark/src/main/scala/SimpleApp.scala:11: value reducePrime is not a member of org.apache.spark.rdd.RDD[Int]
  [error]     logData.reducePrime(_+_);
  [error]             ^
  [error] one error found
  [error] (compile:compileIncremental) Compilation failed
  [error] Total time: 24 s, completed Apr 11, 2015 2:24:03 AM
  </code></pre>
  
  <p><strong>UPDATE</strong>
  Here is the updated sbt file</p>
  
  <pre><code>name := "N Spark"
  
  version := "1.0"
  
  scalaVersion := "2.10"
  
  libraryDependencies += "org.apache.spark" % "1.3.0"
  </code></pre>
  
  <p>I get the following error for this file:</p>
  
  <pre><code>[info] Loading global plugins from /Users/Raggy/.sbt/0.13/plugins
  /Users/Raggy/Downloads/spark-proj/noah-spark/simple.sbt:7: error: No implicit for Append.Value[Seq[sbt.ModuleID], sbt.impl.GroupArtifactID] found,
    so sbt.impl.GroupArtifactID cannot be appended to Seq[sbt.ModuleID]
  libraryDependencies += "org.apache.spark" % "1.3.0"
  </code></pre>