title: How to implement custom job listener/tracker in Spark?
body: <p>I have a class like below, and when i run this through command line i want to see progress status. some thing like, </p>
  
  <pre><code>10% completed... 
  30% completed... 
  100% completed...Job done!
  </code></pre>
  
  <p>I am using spark 1.0 on yarn and using Java API. </p>
  
  <pre><code>public class MyJavaWordCount {
      public static void main(String[] args) throws Exception {
          if (args.length &lt; 2) {
              System.err.println("Usage: MyJavaWordCount &lt;master&gt; &lt;file&gt;");
              System.exit(1);
          }
          System.out.println("args[0]: &lt;master&gt;="+args[0]);
          System.out.println("args[1]: &lt;file&gt;="+args[1]);
  
          JavaSparkContext ctx = new JavaSparkContext(
                  args[0],
                  "MyJavaWordCount",
                  System.getenv("SPARK_HOME"),
                  System.getenv("SPARK_EXAMPLES_JAR"));
          JavaRDD&lt;String&gt; lines = ctx.textFile(args[1], 1);
  
  //      output                                            input   output         
          JavaRDD&lt;String&gt; words = lines.flatMap(new FlatMapFunction&lt;String, String&gt;() {
              //              output       input 
              public Iterable&lt;String&gt; call(String s) {
                  return Arrays.asList(s.split(" "));
              }
          });
  
  //          K       V                                                input   K       V 
          JavaPairRDD&lt;String, Integer&gt; ones = words.mapToPair(new PairFunction&lt;String, String, Integer&gt;() {
              //            K       V             input 
              public Tuple2&lt;String, Integer&gt; call(String s) {
                  //                K       V 
                  return new Tuple2&lt;String, Integer&gt;(s, 1);
              }
          });
  
          JavaPairRDD&lt;String, Integer&gt; counts = ones.reduceByKey(new Function2&lt;Integer, Integer, Integer&gt;() {
              public Integer call(Integer i1, Integer i2) {
                  return i1 + i2;
              }
          });
  
          List&lt;Tuple2&lt;String, Integer&gt;&gt; output = counts.collect();
          for (Tuple2 tuple : output) {
              System.out.println(tuple._1 + ": " + tuple._2);
          }
          System.exit(0);
      }
  }
  </code></pre>