title: Writing a large number of queries to a text file
body: <p>I have a list of about 200,000 entities, and I need to query a specific RESTful API for each of those entities, and end up with all the 200,000 entities saved in JSON format in txt files.
  The naive way of doing it is going through the list of the 200,000 entities and query one by one, add the returned JSON to a list, and when it's done, right all to a text file. Something like:</p>
  
  <pre><code>from apiWrapper import api
  from entities import listEntities #list of the 200,000 entities
  a=api()
  fullEntityList=[]
  for entity in listEntities:
  fullEntityList.append(a.getFullEntity(entity))
  
  with open("fullEntities.txt","w") as f:
      simplejson.dump(fullEntityList,f)
  </code></pre>
  
  <p>Obviously this is not reliable, as 200,000 queries to the API will take about 10 hours or so, so I guess something will cause an error before it gets to write it to the file.
  I guess the right way is to write it in chunks, but not sure how to implement it. Any ideas?
  Also, I cannot do this with a database.</p>