title: What is a sensible way to store matrices (which represent images) either in memory or on disk, to make them available to a GUI application?
body: <p>I am looking for some high level advice about a project that I am attempting.</p>
  <p>I want to write a PyQt application (following the <a href="https://doc.qt.io/qtforpython/overviews/model-view-programming.html" rel="nofollow noreferrer">model-view pattern</a>) to read in images from a directory one by one and process them. Typically there will be a few thousand .png images (each around 1 megapixel, 16 bit grayscale) in the directory. After being read in, the application will then process the integer pixel values of each image in some way, and crucially the result will be a matrix of floats for each. Once processed, the user should be able be able to then go back and explore any of the matrices they choose (or multiple at once), and possibly apply further processing.</p>
  <p>My question is regarding a sensible way to store the matrices in memory, and access them when needed. After reading in the raw .png files and obatining the corresponding matrix of floats, I can then see the following options for handling the result:</p>
  <ol>
  <li><p>Simply <strong>store each matrix as a numpy array and have every one of them stored in a class attribute</strong>. That way they will all be easily accessible to the code when requested by the user, but will this be poor in terms of RAM required?</p>
  </li>
  <li><p>After processing each, <strong>write out the matrix to a text file</strong>, and read it back in from the text file when requested by the user.</p>
  </li>
  <li><p>I have seen examples (<a href="https://www.giacomodebidda.com/mvc-pattern-in-python-sqlite/" rel="nofollow noreferrer">see here</a>) of people using <strong>SQLite databases</strong> to store data for a GUI application (using MVC pattern), and then query the database when you need access to data. This seems like it would have the advantage that data is not stored in RAM by the &quot;model&quot; part of the application (like in option 1), and is possibly more storage-efficient than option 2, but is this suitable given that my data are matrices?</p>
  </li>
  <li><p>I have seen examples (<a href="https://www.pythonforthelab.com/blog/how-to-use-hdf5-files-in-python/" rel="nofollow noreferrer">see here</a>) of people using something called <strong>HDF5</strong> for storing application data, and that this might be similar to using a SQLite database? Again, suitable for matrices?</p>
  </li>
  <li><p>Finally, I see that PyQt has the classes <strong><a href="https://doc.qt.io/qt-5/qimage.html" rel="nofollow noreferrer">QImage</a> and <a href="https://doc.qt.io/qt-5/qpixmap.html" rel="nofollow noreferrer">QPixmap</a></strong>. Do these make sense for solving the problem I have described?</p>
  </li>
  </ol>
  <p>I am a little lost with all the options, and don't want to spend too much time investigating <em>all</em> of them in too much detail so would appreciate some general advice. If someone could offer comments on each of the options I have described (as well as letting me know if any can be ruled out in this situation) that would be great!</p>
  <p>Thank you</p>