title: Hive Dynamic Partition with Multiple Files in Each Partition
body: <p>This is how am dynamically partitioning the data, but only a single file is getting created per partition but I want each file to be of 256MB in size.</p>
  
  <p>And since my data is heavily skewed, some files associated with Partitions are 2-3 GB in size rightnow... I tried so many settings, but none of them worked.</p>
  
  <p>Am using MR mode.</p>
  
  <p>Can someone please let me know what could be done here to create multiple files per partitions where each file should be of 256MB in size ?</p>
  
  <pre><code>SET hive.execution.engine=mr;
  SET hive.exec.scratchdir=/tmp/hive/ ;
  SET hive.exec.stagingdir=/tmp/hive/;
  
  set hive.msck.path.validation=ignore;
  SET hive.exec.dynamic.partition = true;
  SET hive.exec.dynamic.partition.mode = nonstrict;
  set hive.exec.reducers.bytes.per.reducer=250000000;
  set mapreduce.reduce.shuffle.memory.limit.percent=0.15;
  
  insert overwrite table madlytics.events_eors20_hive_dynamic_part partition(h='01',event_type)
  select
  *, h, event_type
  from
  madlytics.events_eors_2017_09_20 where h = '01' distribute by event_id sort by event_type;
  </code></pre>