title: "Your input ran out of data" always at step # of epochs
body: <p>I have been trying to make my first data generator for a model.fit() with Keras. The dataset I'm trying to make has two inputs, an image and a float value. All of my image names and values are stored in a csv file. I believe I made my generator incorrectly because <strong>no matter what my batch size is, I always get the error &quot;Your input ran out of data&quot; at the step equal to my epochs</strong>. So if my epochs are set to 100 my model will run until it reaches step 100. My dataset is about 100000 images/values big. If anyone could help me find a solution that would be great.<br />
  I am currently using:<br />
  python 3.8<br />
  tf-gpu 2.4.0rc1<br />
  keras 2.4.3<br />
  pandas 1.1.4</p>
  <p>Code:</p>
  <pre><code>IMG_SIZE = 400
  Version = 1
  batch_size = 64
  val = .05
  
  val_aug = ImageDataGenerator(rescale=1/255)
  aug = ImageDataGenerator(
          rescale=1/255, 
          rotation_range=30, 
          width_shift_range=0.1, 
          height_shift_range=0.1, 
          shear_range=0.2, 
          zoom_range=0.2, 
          channel_shift_range=25, 
          horizontal_flip=True, 
          fill_mode='constant')
  
  
  df = pd.read_csv('F:/DATA/Vote/Vote_Age.csv')
  
  df = df.sample(frac = 1)
  cut = int(len(df) * val)
  
  train_df = df[cut:]
  val_df = df[0:cut]
  print(f'Training dataset: {len(train_df)}')
  print(f'Val dataset: {len(val_df)}')
  
  train_steps = int(len(train_df) / batch_size)
  val_steps = int(len(val_df) / batch_size)
  
  def data(df, generator, batch_size, IMG_SIZE):
          z = 0
          while True:
                  df = df.sample(frac = 1)
                  for i in range(int(len(df) / batch_size)):
                          images, ages, votes = [], [], []
                          for x in range(batch_size):
                                  csv_row = df.iloc[(z), :]
                                  z += 1
                                  image_path = f'F:/DATA/Vote/Images/{int(csv_row[0])}.jpg'
                                  image = cv2.resize(cv2.imread(image_path), (int(IMG_SIZE), int(IMG_SIZE)))
                                  image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
                                  image = image.reshape(-1, IMG_SIZE, IMG_SIZE, 3)
                                  generator.fit(image)
                                  image = generator.flow(image, batch_size=1)
                                  image = image.next()
                                  image = image.reshape(IMG_SIZE, IMG_SIZE, 3)
  
                                  images.append(image)
                                  ages.append(csv_row[1])
                                  votes.append(int(csv_row[2]))
  
                          images = np.array(images)
                          ages = np.array(ages)
                          votes = np.array(votes)
                  
                          return [[images, ages], [votes]]
  
  #########
  #Model was very big and unnecessary to include
  #########
  
  train_dataset = train_data(train_df, batch_size, IMG_SIZE)
  val_dataset = val_data(val_df, batch_size, IMG_SIZE)
  
  model.fit(
      x = train_dataset[0],
      y = train_dataset[1],
      validation_data=(val_dataset[0], val_dataset[1]),
      steps_per_epoch=train_steps,
      validation_steps=val_steps,
      callbacks=earlyStop,
      epochs=100, batch_size=batch_size,
      workers=multiprocessing.cpu_count(),
      verbose=1)
  
  model.save(f'F:/DATA/Vote/Models/YiffModel{Version}')
  </code></pre>
  <p>Ouput:</p>
  <pre><code>...
    83/1484 [&gt;.............................] - ETA: 4:46 - loss: 7578.7731
    84/1484 [&gt;.............................] - ETA: 4:46 - loss: 7575.5172
    85/1484 [&gt;.............................] - ETA: 4:46 - loss: 7572.2818
    86/1484 [&gt;.............................] - ETA: 4:46 - loss: 7569.0662
    87/1484 [&gt;.............................] - ETA: 4:46 - loss: 7565.8702
    88/1484 [&gt;.............................] - ETA: 4:45 - loss: 7562.6932
    89/1484 [&gt;.............................] - ETA: 4:45 - loss: 7559.5349
    90/1484 [&gt;.............................] - ETA: 4:45 - loss: 7556.3948
    91/1484 [&gt;.............................] - ETA: 4:45 - loss: 7553.2726
    92/1484 [&gt;.............................] - ETA: 4:44 - loss: 7550.1679
    93/1484 [&gt;.............................] - ETA: 4:44 - loss: 7547.0802
    94/1484 [&gt;.............................] - ETA: 4:44 - loss: 7544.0094
    95/1484 [&gt;.............................] - ETA: 4:44 - loss: 7540.9549
    96/1484 [&gt;.............................] - ETA: 4:43 - loss: 7537.9164
    97/1484 [&gt;.............................] - ETA: 4:43 - loss: 7534.8937
    98/1484 [&gt;.............................] - ETA: 4:43 - loss: 7531.8863
    99/1484 [=&gt;............................] - ETA: 4:43 - loss: 7528.8939
   100/1484 [=&gt;............................] - ETA: 4:43 - loss: 7525.9163
  WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 148400 batches). You may need to use the repeat() function when building your dataset.
  WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 78 batches). You may need to use the repeat() function when building your dataset.
  
  1484/1484 [==============================] - 30s 15ms/step - loss: 7250.9988 - val_loss: 13595.9355
  C:\Users\Tristan\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\training.py:2325: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
    warnings.warn('`Model.state_updates` will be removed in a future version. '
  2021-01-05 15:37:04.425018: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
  C:\Users\Tristan\anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:1402: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
    warnings.warn('`layer.updates` will be removed in a future version. '
  WARNING:tensorflow:FOR KERAS USERS: The object that you are saving contains one or more Keras models or layers. If you are loading the SavedModel with `tf.keras.models.load_model`, continue reading (otherwise, you may ignore the following instructions). Please change your code to save with `tf.keras.models.save_model` or `model.save`, and confirm that the file &quot;keras.metadata&quot; exists in the export directory. In the future, Keras will only load the SavedModels that have this file. In other words, `tf.saved_model.save` will no longer write SavedModels that can be recovered as Keras models (this will apply in TF 2.5).
  
  FOR DEVS: If you are overwriting _tracking_metadata in your class, this property has been used to save metadata in the SavedModel. The metadta field will be deprecated soon, so please move the metadata to a different file.
  libpng warning: iCCP: known incorrect sRGB profile
  </code></pre>