title: Why Hadoop MapReduce Only Launch 1 Reducer
body: <p>I tried to distinct 2GB data (JSON type, 1 record is 1 JSON Object) using MapReduce in a cluster of 4 computer. Mapping works just fine, there is 17 map task launch (since my block size is 128MB) but there is only 1 reduce task launch. I'm not setting number of reducer in my code. </p>
  
  <p>Mapper code</p>
  
  <pre><code>public static class DistinctMapper extends
          Mapper&lt;Object, Text, Text, Text&gt; {
  
      private Text outFieldKey = new Text();
      private String field;
  
      @Override
      protected void setup(Context context) throws IOException,
              InterruptedException {
          this.field = context.getConfiguration().get("field");
      }
  
      @Override
      public void map(Object key, Text value, Context context)
              throws IOException, InterruptedException {
  
          JSONObject parsed = new JSONObject();
          String strValue = value.toString();
          parsed = new JSONObject(strValue);
  
          String selectedFieldVal = "";
          selectedFieldVal = parsed.get(this.field).toString();
  
          outFieldKey.set(selectedFieldVal);
          context.write(outFieldKey, value);
      }
  }
  </code></pre>
  
  <p>Combiner Code</p>
  
  <pre><code>public static class DistinctCombiner extends
          Reducer&lt;Text, Text, Text, Text&gt; {
  
      public void reduce(Text key, Iterable&lt;Text&gt; values,
              Reducer.Context context) throws IOException, InterruptedException {
          context.write(key, values.iterator().next());
      }
  }
  </code></pre>
  
  <p>Reducer Code</p>
  
  <pre><code> public static class DistinctReducer extends
          Reducer&lt;Text, Text, Text, Text&gt; {
  
      @Override
      public void reduce(Text key, Iterable&lt;Text&gt; values,
              Context context) throws IOException, InterruptedException {
          context.write(new Text(), values.iterator().next());
      }
  }
  </code></pre>
  
  <p>Main</p>
  
  <pre><code>public static void main(String[] args) throws Exception {
      Configuration conf = new Configuration();
      String[] otherArgs = new GenericOptionsParser(conf, args)
              .getRemainingArgs();
      if (otherArgs.length != 3) {
          System.err.println("Usage: DistinctDataBySomeField &lt;field&gt; &lt;in&gt; &lt;out&gt;");
          System.exit(2);
      }
  
      conf.set("field", otherArgs[0]);
      Job job = new Job(conf, "Distinct data by field x");
      job.setJarByClass(DistinctPatternDriver.class);
      job.setMapperClass(DistinctMapper.class);
      job.setCombinerClass(DistinctCombiner.class);
      job.setReducerClass(DistinctReducer.class);
      job.setOutputKeyClass(Text.class);
      job.setOutputValueClass(Text.class);
      FileInputFormat.addInputPath(job, new Path(otherArgs[1]));
      FileOutputFormat.setOutputPath(job, new Path(otherArgs[2]));
      System.exit(job.waitForCompletion(true) ? 0 : 1);
  }
  </code></pre>
  
  <p>I need many reducer so it can be faster. Why is it happen? </p>