title: Ingesting HDF5 data: keys within keys, getting data into arrays
body: <p>I need some help.</p>
  <p>I have an HDF5 file containing spectrum data (time, frequencies, and the power level of a given frequency at a given time). Here is how the file structured (using HDFView):</p>
  <p><a href="https://i.stack.imgur.com/0UZZS.png" rel="nofollow noreferrer">HDFView of Data File</a></p>
  <p>The main groups (keys) are the HOURS, and then inside are the minutes, with each minute being its own group (key). Data was collected at 0.02 seconds for 60 seconds... so there are 3000 rows...and the frequency bins are 256 (i.e. start at 1MHz and end at 26MHz, spaced into 256 spacing apart. For example.e. 23 --&gt; 23:10 --&gt; 2D array of the power</p>
  <pre><code>0           0           1           2       .....           255
  1       -53.672386  -53.82235   -53.773468  .....        -50.566887
  2       -53.85694   -53.945183  -53.63385   .....        -51.306465   
  3       -53.709038  -53.55101   -53.55305   .....        -52.7324906
  .
  .
  .
  2999    -53.23989   -51.501495  -50.681602              -52.227474
  </code></pre>
  <p>I am able to access the individual minutes data and pull them into arrays and then plot the data. Like this:</p>
  <pre><code>import h5py
  import numpy as np
  import matplotlib.pyplot as plt
  
  # Read in the HDF5 file
  file = h5py.File(&quot;/home/tom/Desktop/2021-10-28_ch0.hdf5&quot;, 'r')
  
  # Search for the main groups in the file. The main groups are hours: 20, 22, etc...
  
  # Select one of the hours (i.e. 23)
  hour = file['23']
  
  # Search for the subgroups (keys) within the chosen hour. There are &quot;hour:minutes&quot; i.e. 23:10
  #for key in hour.keys():
      #print( key )
  
  # Select key with data for minutes 10, 11, 12, 13 and save into individual arrays:
  minute_data_10=hour['23:10'][()]
  minute_data_11=hour['23:11'][()]
  minute_data_12=hour['23:12'][()]
  minute_data_13=hour['23:13'][()]
  
  # Generate a 1D array of TIME spanning 4 minutes (because we ingested
  # 4x 1 minute slices of data:
  
  time = np.linspace(0, 60*4, 3000*4)
  
  # Generate a 1D array of FREQUENCY
  frequency = np.linspace(1.575E0, 26.82402336E0, 256)
  
  # Combine minute_data_10  minute_data_11 minute_data_12 and minute_data_13 along the time axis (axis=0)
  comb_min = np.concatenate( (minute_data_10, minute_data_11, minute_data_12, minute_data_13), axis=0 )
  
  print( comb_min.shape )
  
  # Plot the data
  im = plt.pcolormesh(frequency, time, comb_min, cmap='jet')
  plt.colorbar(im).ax.tick_params(labelsize=10)
  plt.title('Spectrum')
  plt.ylabel('Seconds ago...')
  #plt.xlabel('frequency in Hz')
  im.axes.xaxis.set_ticklabels([])
  plt.show()
  </code></pre>
  <p><a href="https://i.stack.imgur.com/kLHln.png" rel="nofollow noreferrer">Spectrum Plotted</a></p>
  <p>I am manually defining each minute (min 10, 11, 12, 13) combining them and then plotting them.</p>
  <p>BUT...what I would like to do is to automatically ingest ALL minutes for ALL hours of my choosing and then plot it into one plot. For example, how can I ingest ALL minutes in hour 15 and then plot the spectrum ? OR, how could I plot the first 5 hours of the data ?</p>