title: Reading Large HDF5 Files
body: <p>I am new to using HDF5 files and I am trying to read files with shapes of <code>(20670, 224, 224, 3)</code>. Whenever I try to store the results from the hdf5 into a list or another data structure, it takes either takes so long that I abort the execution or it crashes my computer. I need to be able to read 3 sets of hdf5 files, use their data, manipulate it, use it to train a CNN model and make predictions.</p>
  <p>Any help for reading and using these large HDF5 files would be greatly appreciated.</p>
  <p>Currently this is how I am reading the hdf5 file:</p>
  <pre><code>db = h5py.File(os.getcwd() + &quot;/Results/Training_Dataset.hdf5&quot;)
  training_db = list(db['data'])
  </code></pre>