title: Iteratively writing to HDF5 Stores in Pandas
body: <p><a href="http://pandas.pydata.org/pandas-docs/dev/io.html#notes-caveats">Pandas</a> has the following examples for how to store <code>Series</code>, <code>DataFrames</code> and <code>Panels</code>in HDF5 files:</p>
  
  <h2>Prepare some data:</h2>
  
  <pre><code>In [1142]: store = HDFStore('store.h5')
  
  In [1143]: index = date_range('1/1/2000', periods=8)
  
  In [1144]: s = Series(randn(5), index=['a', 'b', 'c', 'd', 'e'])
  
  In [1145]: df = DataFrame(randn(8, 3), index=index,
     ......:                columns=['A', 'B', 'C'])
     ......:
  
  In [1146]: wp = Panel(randn(2, 5, 4), items=['Item1', 'Item2'],
     ......:            major_axis=date_range('1/1/2000', periods=5),
     ......:            minor_axis=['A', 'B', 'C', 'D'])
     ......:
  </code></pre>
  
  <h2>Save it in a store:</h2>
  
  <pre><code>In [1147]: store['s'] = s
  
  In [1148]: store['df'] = df
  
  In [1149]: store['wp'] = wp
  </code></pre>
  
  <h2>Inspect what's in the store:</h2>
  
  <pre><code>In [1150]: store
  Out[1150]: 
  &lt;class 'pandas.io.pytables.HDFStore'&gt;
  File path: store.h5
  /df            frame        (shape-&gt;[8,3])  
  /s             series       (shape-&gt;[5])    
  /wp            wide         (shape-&gt;[2,5,4])
  </code></pre>
  
  <h1>Close the store:</h1>
  
  <pre><code>In [1151]: store.close()
  </code></pre>
  
  <h2>Questions:</h2>
  
  <ol>
  <li><p>In the code above, <strong>when is the data actually written to disk</strong>? </p></li>
  <li><p>Say I want to add thousands of large dataframes living in <code>.csv</code> files to a single <code>.h5</code> file. I would need to load them and add them to the <code>.h5</code> file one by one since I <strong>cannot</strong> afford to have them all in memory at once as they would take too much memory. Is this possible with HDF5? What would be the correct way to do it?</p></li>
  <li><p>The Pandas documentation says the following: </p>
  
  <blockquote>
    <p>"These stores are not appendable <strong>once written</strong> (though you simply
    remove them and rewrite). Nor are they <strong>queryable</strong>; they must be
    retrieved in their entirety."</p>
  </blockquote>
  
  <p>What does it mean by <strong>not appendable nor queryable</strong>? Also, shouldn't it say once <strong>closed</strong> instead of <strong>written</strong>?</p></li>
  </ol>