title: confused about the concept of timesteps and output shape in keras.layers.RNN
body: <blockquote>
    <p>keras.layers.RNN</p>
    
    <p>Input shape
    3D tensor with shape (batch_size, timesteps, input_dim).</p>
    
    <p>Output shape</p>
    
    <p>if return_state: a list of tensors. The first tensor is the output. The remaining tensors are the last states, each with shape  (batch_size, units).</p>
    
    <p>if return_sequences: 3D tensor with shape  (batch_size, timesteps, units).
    else, 2D tensor with shape (batch_size, units).</p>
  </blockquote>
  
  <p>1.I am comfused about the concept of timesteps.</p>
  
  <p>2.I am confused about the process that how  it deals with the input of three axes.</p>
  
  <h1>Simplified code</h1>
  
  <pre><code>import keras
  from keras.applications.inception_resnet_v2 import InceptionResNetV2
  from keras.applications.mobilenet import MobileNet
  from keras.applications.vgg19 import VGG19
  from keras.applications.densenet import DenseNet
  from keras.preprocessing import image
  from keras.engine import Layer
  from keras.applications.inception_resnet_v2 import preprocess_input
  from keras.layers import Conv2D, UpSampling2D, InputLayer, Conv2DTranspose, Input, Reshape, merge, concatenate
  from keras.layers import Activation, Dense, Dropout, Flatten
  from keras.layers.normalization import BatchNormalization
  from keras.callbacks import TensorBoard 
  from keras.models import Sequential, Model
  from keras.layers.core import RepeatVector, Permute
  from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
  from skimage.color import rgb2lab, lab2rgb, rgb2gray, gray2rgb
  from skimage.transform import resize
  from skimage.io import imsave
  import numpy as np
  import os
  import random
  import tensorflow as tf
  import keras.backend as K
  from keras.layers.recurrent import GRU
  from keras.layers.merge import add
  
  encoder_input = Input(shape=(32, 32, 1))
  
  rnn_size = 16
  encoder_output = Conv2D(16, (3,3), activation='relu', padding='same')(encoder_input)
  sequence_output = Reshape(target_shape=(32, 512))(encoder_output)  
  gru_1 = GRU(rnn_size, return_sequences=False,kernel_initializer='he_normal', name='gru1')(sequence_output)
  gru_1b = GRU(rnn_size, return_sequences=True, go_backwards=True, kernel_initializer='he_normal', name='gru1_b')(sequence_output)
  gru1_merged = add([gru_1, gru_1b])
  gru_2 = GRU(rnn_size, return_sequences=True,kernel_initializer='he_normal', name='gru2')(gru1_merged)
  gru_2b = GRU(rnn_size, return_sequences=True, go_backwards=True, kernel_initializer='he_normal', name='gru2_b')(gru1_merged)
  sequence_output = concatenate([gru_2, gru_2b])
  sequence_output = K.expand_dims(sequence_output, 3)
  fusion_output = concatenate([encoder_output,sequence_output ], axis=3) 
  
  model = Model(inputs=encoder_input, outputs=fusion_output)
  model.summary()
  </code></pre>
  
  <h1>output error</h1>
  
  <blockquote>
    <p>ValueError: A <code>Concatenate</code> layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 32, 16), (None, None, 32, 1)]</p>
  </blockquote>
  
  <p>I thought the shape of 'sequence_output' would be (None, 32, 32, 1).But it was (None, None, 32, 1).I didn't know what went wrong so I started to doubt my understanding about RNN.</p>
  
  <h1>What I have done</h1>
  
  <h1>1.output gru1 and gru_1b</h1>
  
  <p>after annotation:</p>
  
  <p>'#sequence_output = K.expand_dims(sequence_output, 3)'</p>
  
  <p>'#fusion_output = concatenate([encoder_output,sequence_output ], axis=3)'
  Then I got ()<a href="https://i.stack.imgur.com/UckkM.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UckkM.png" alt="output"></a></p>
  
  <h3>I am quite confused about the shape of gru1 and gru_1b. Why they are different?</h3>
  
  <h1>2.I set return_sequences and return_state True and then got</h1>
  
  <p>valueerror:
  <a href="https://i.stack.imgur.com/2O2Yx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2O2Yx.png" alt=" "></a>
  In fact I have no idea what to do next.</p>