title: Reading csv file in chunks for processing
body: <p>I have a .csv file with 100 000 records with five columns in it. I am reading it line by line and storing it in a remote database .</p>
  
  <p>Previously, I was following a performance oriented approach. I was reading the .csv file line by line and in the same transaction I was opening the connection to database and closing it. This was taking a serious performance overhead.
  For just writing 10 000 lines, it took one hour.</p>
  
  <pre><code>using (FileStream reader = File.OpenRead(@"C:\Data.csv")) 
              using (TextFieldParser parser = new TextFieldParser(reader))
              {
                  parser.TrimWhiteSpace = true; // if you want
                  parser.Delimiters = new[] { " " };
                  parser.HasFieldsEnclosedInQuotes = true;
  
                  while (!parser.EndOfData)
                  {
                      //Open a connection to a database 
                      //Write the data from the .csv file line by line
                      //Close the connection
                   }
               }
  </code></pre>
  
  <p>Now I have changed the approach. For testing purpose I have taken a .csv file with 10 000 lines and after reading all the 10 000 lines, I am making one connection to database and writing it there.</p>
  
  <p>Now, the only issue is:
  I want to read first 10 000 lines and write it, similarly read the next 10 000 lines and write it, </p>
  
  <pre><code>using (FileStream reader = File.OpenRead(@"C:\Data.csv")) 
                  using (TextFieldParser parser = new TextFieldParser(reader))
  </code></pre>
  
  <p>but the above two lines will read the entire file . I don’t want to read it completely.
  Is there any way to read the .csv file chunk by chunk of 10 000 lines each?</p>