title: Reading large number of images in python using opencv
body: <p>I am making a food classification program using machine learning models.I have avery large data set of 101 classes and each class has around 1000 images but working with such a large dataset has caused me to run out of memory hence I used a smaller data set and I found the classifier to be biased for one training class.I am using cv2.imread() to read the images and then I have used scikit to train and test the models. my code is as follows </p>
  
  <pre><code>def image_to_feature_vector(image, size=(128, 128)):
      # resize the image to a fixed size, then flatten the image into
      # a list of raw pixel intensities
      return cv2.resize(image, size).flatten()
  
  def extract_color_histogram(image, bins=(32, 32, 32)):
      # extract a 3D color histogram from the HSV color space using
      # the supplied number of `bins` per channel
      hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
      hist = cv2.calcHist([hsv], [0, 1, 2], None, bins,
          [0, 180, 0, 256, 0, 256])
  
      # handle normalizing the histogram if we are using OpenCV 2.4.X
      if imutils.is_cv2():
          hist = cv2.normalize(hist)
  
      # otherwise, perform "in place" normalization in OpenCV 3
      else:
          cv2.normalize(hist, hist)
  
      # return the flattened histogram as the feature vector
      return hist.flatten()
  
  
  
  class_names=[]
  read_images = []
  # initialize the raw pixel intensities matrix, the features matrix,
  # and labels list
  rawImages = []
  features = []
  folders = glob.glob('E:\\food\\images\\*')
  imagenames__list = []
  for folder in folders:
      for f in glob.glob(folder+'/*.jpg'):
          imagenames__list.append(f)
          image=cv2.imread(f)
          #plt.imshow(image)
          a=os.path.basename(folder)
          class_names.append(a)
  
  
          pixels = image_to_feature_vector(image)
          hist = extract_color_histogram(image)# add the messages we got to the raw images, features matricies
          rawImages.append(pixels)
          features.append(hist)
  
  
  
  print(class_names)
  print(f)            
  rawImages = np.array(rawImages)
  print(len(rawImages))
  features = np.array(features)
  print(len(rawImages))
  class_names = np.array(class_names)
  </code></pre>
  
  <p>Can you suggest any changes to the code so That I can use the full dataset.Please point out if this approach is completely wrong and how do I change it. </p>