title: Using pandas to efficiently read in a large CSV file without crashing
body: <p>I am trying to read a .csv file called ratings.csv from <a href="http://grouplens.org/datasets/movielens/20m/" rel="nofollow noreferrer">http://grouplens.org/datasets/movielens/20m/</a> the file is 533.4MB in my computer.</p>
  
  <p>This is what am writing in jupyter notebook</p>
  
  <pre><code>import pandas as pd
  ratings = pd.read_cv('./movielens/ratings.csv', sep=',')
  </code></pre>
  
  <p>The problem from here is the kernel would break or die and ask me to restart and its keeps repeating the same. There is no any error. Please can you suggest any alternative of solving this, it is as if my computer has no capability of running this. </p>
  
  <p>This works but it keeps rewriting</p>
  
  <pre><code>chunksize = 20000
  for ratings in pd.read_csv('./movielens/ratings.csv', chunksize=chunksize):
  ratings.append(ratings)
  ratings.head()
  </code></pre>
  
  <p>Only the last chunk is written others are written-off</p>