title: Spring Batch - How to reads 5 million records in faster ways?
body: <p>I'm developing Spring Boot v2.2.5.RELEASE and Spring Batch example. In this example, I'm reading 5 million records using <code>JdbcPagingItemReader</code> from <code>Postgres</code> system from one data-center and writing in into <code>MongoDB</code> into another data-center.  </p>
  
  <p>This migration is too slow and need to make the better performance of this batch job. I 'm not sure on how to use partition, because I have a PK in that table holds UUID values, so I can't think of using <code>ColumnRangePartitioner</code>. Is there any best approach to implement this?</p>
  
  <p>Approach-1:</p>
  
  <pre><code>@Bean
  public JdbcPagingItemReader&lt;Customer&gt; customerPagingItemReader(){
      // reading database records using JDBC in a paging fashion
      JdbcPagingItemReader&lt;Customer&gt; reader = new JdbcPagingItemReader&lt;&gt;();
      reader.setDataSource(this.dataSource);
      reader.setFetchSize(1000);
      reader.setRowMapper(new CustomerRowMapper());
  
      // Sort Keys
      Map&lt;String, Order&gt; sortKeys = new HashMap&lt;&gt;();
      sortKeys.put("cust_id", Order.ASCENDING);
  
      // POSTGRES implementation of a PagingQueryProvider using database specific features.
      PostgresPagingQueryProvider queryProvider = new PostgresPagingQueryProvider();
      queryProvider.setSelectClause("*");
      queryProvider.setFromClause("from customer");
      queryProvider.setSortKeys(sortKeys);
  
      reader.setQueryProvider(queryProvider);
  
      return reader;
  }
  </code></pre>
  
  <p>Then Mongo writer, I've used Spring Data Mongo as custom writer:</p>
  
  <p>Job details</p>
  
  <pre><code>@Bean
      public Job multithreadedJob() {
          return this.jobBuilderFactory.get("multithreadedJob")
                  .start(step1())
                  .build();
      }
  
      @Bean
      public Step step1() {
          ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
          taskExecutor.setCorePoolSize(4);
          taskExecutor.setMaxPoolSize(4);
          taskExecutor.afterPropertiesSet();
  
          return this.stepBuilderFactory.get("step1")
                  .&lt;Transaction, Transaction&gt;chunk(100)
                  .reader(fileTransactionReader(null))
                  .writer(writer(null))
                  .taskExecutor(taskExecutor)
                  .build();
      }
  </code></pre>
  
  <p>Approach-2: AsyncItemProcessor and AsyncItemWriter would be the better option, because still I've to read using same JdbcPagingItemReader?</p>
  
  <p>Approach-3: Partition, how to use it where I've PK as UUID?</p>