title: Renaming Part Files in Hadoop Map Reduce
body: <p>I have tried to use the <code>MultipleOutputs</code> class as per the example in page <a href="http://hadoop.apache.org/docs/mapreduce/r0.21.0/api/index.html?org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html">http://hadoop.apache.org/docs/mapreduce/r0.21.0/api/index.html?org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html</a></p>
  
  <p>Driver Code</p>
  
  <pre><code>    Configuration conf = new Configuration();
      Job job = new Job(conf, "Wordcount");
      job.setJarByClass(WordCount.class);
      job.setInputFormatClass(TextInputFormat.class);
      job.setMapperClass(WordCountMapper.class);
      job.setReducerClass(WordCountReducer.class);
      job.setMapOutputKeyClass(Text.class);
      job.setMapOutputValueClass(IntWritable.class);
      job.setOutputKeyClass(Text.class);
      job.setOutputValueClass(IntWritable.class);
      FileInputFormat.setInputPaths(job, new Path(args[0]));
      FileOutputFormat.setOutputPath(job, new Path(args[1]));
      MultipleOutputs.addNamedOutput(job, "text", TextOutputFormat.class,
              Text.class, IntWritable.class);
      System.exit(job.waitForCompletion(true) ? 0 : 1);
  </code></pre>
  
  <p>Reducer Code</p>
  
  <pre><code>public class WordCountReducer extends
          Reducer&lt;Text, IntWritable, Text, IntWritable&gt; {
      private IntWritable result = new IntWritable();
      private MultipleOutputs&lt;Text, IntWritable&gt; mos;
      public void setup(Context context){
          mos = new MultipleOutputs&lt;Text, IntWritable&gt;(context);
      }
      public void reduce(Text key, Iterable&lt;IntWritable&gt; values, Context context)
              throws IOException, InterruptedException {
          int sum = 0;
          for (IntWritable val : values) {
              sum += val.get();
          }
          result.set(sum);
          //context.write(key, result);
          mos.write("text", key,result);
      }
      public void cleanup(Context context)  {
           try {
              mos.close();
          } catch (IOException e) {
              // TODO Auto-generated catch block
              e.printStackTrace();
          } catch (InterruptedException e) {
              // TODO Auto-generated catch block
              e.printStackTrace();
          }
           }
  }
  </code></pre>
  
  <p>The output of the reducer is found to rename to text-r-00000</p>
  
  <p>But the issue here is that I am also getting an empty part-r-00000 file as well. Is this how MultipleOutputs is expected to behave, or is there some problem with my code? Please advice.</p>
  
  <p>Another alternative I have tried out is to iterate through my output folder using the FileSystem  class and manually rename all files beginning with part.</p>
  
  <p>What is the best way? </p>
  
  <pre><code>FileSystem hdfs = FileSystem.get(configuration);
  FileStatus fs[] = hdfs.listStatus(new Path(outputPath));
  for (FileStatus aFile : fs) {
  if (aFile.isDir()) {
  hdfs.delete(aFile.getPath(), true);
  // delete all directories and sub-directories (if any) in the output directory
  } 
  else {
  if (aFile.getPath().getName().contains("_"))
  hdfs.delete(aFile.getPath(), true);
  // delete all log files and the _SUCCESS file in the output directory
  else {
  hdfs.rename(aFile.getPath(), new Path(myCustomName));
  }
  }
  </code></pre>