title: how to spark-submit job to spark cluster on local kubernetes created by minikube
body: <p>I am reading the <a href="http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-6/" rel="nofollow noreferrer">blog</a> and tried to run the <a href="https://github.com/phatak-dev/kubernetes-spark" rel="nofollow noreferrer">codes</a>.</p>
  
  <pre><code>$kubectl get po
  NAME                            READY     STATUS    RESTARTS   AGE
  spark-master-668325562-w369p    1/1       Running   0          23s
  spark-worker-1868749523-xt7hg   1/1       Running   0          23s
  </code></pre>
  
  <p>Now, the spark cluster is running well on local kubernetes cluster created by minikube. I am trying to submit spark job to it by the following command:</p>
  
  <pre><code>spark-2.1.1-bin-hadoop2.7/bin$ ./spark-submit --master spark://&lt;spark-master&gt;:7077 /home/me/workspace/myproj/myproj.jar
  </code></pre>
  
  <p>How to know the spark-master IP? I just followed the above steps to do it and cannot find related tutorials about how to <strong>know/set</strong> the spark-master IP. </p>
  
  <p>Anyone can explain it? Thanks</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>I tried the following ips, but failed.</p>
  
  <pre><code>$ minikube ip
  192.168.42.55
  
  $kubectl get svc 
  NAME           CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
  kubernetes     10.0.0.1     &lt;none&gt;        443/TCP             3h
  spark-master   10.0.0.175   &lt;none&gt;        8080/TCP,7077/TCP   42m
  </code></pre>
  
  <p>Error:</p>
  
  <pre><code>Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
  Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Can only call getServletHandlers on a running MetricsSystem
      at scala.Predef$.require(Predef.scala:224)
      at org.apache.spark.metrics.MetricsSystem.getServletHandlers(MetricsSystem.scala:91)
      at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:524)
      at NetworkScanCounter$.main(network-scan-counter.scala:68)
      at NetworkScanCounter.main(network-scan-counter.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:743)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  </code></pre>