title: keras (lstm) - necessary shape when using return_sequences=True
body: <p>I am trying to fit an LSTM network to a sin function. Currently, as far as I understand Keras, my code does only predict the next value. According to this link: <a href="https://stackoverflow.com/questions/43034960/many-to-one-and-many-to-many-lstm-examples-in-keras">Many to one and many to many LSTM examples in Keras</a> it is a many to one model. However, my goal is to implement a Many-to-many model. Basically, I want to be able to predict let's say 10 values, to a given time. When I am trying to use 
  <code>return_sequences=True</code> (see line <code>model.add(..)</code>), which is supposed to be the solution, the following error occurs:</p>
  
  <pre><code>ValueError: Error when checking target: expected lstm_8 to have 3 dimensions, but got array with shape (689, 1)
  </code></pre>
  
  <p>Unfortunately, I have absolutely no clue why this happens. Is there a general rule how the input shape needs to be when using <code>return_sequences=True</code> ? Furthermore what exactly would I need to change? Thanks for any help.</p>
  
  <pre><code>import pandas
  import numpy as np
  import matplotlib.pylab as plt
  from sklearn.preprocessing import MinMaxScaler
  from sklearn.model_selection import train_test_split
  from sklearn.metrics import mean_squared_error
  import sklearn
  
  from keras.models import Sequential
  from keras.layers import Activation, LSTM
  from keras import optimizers
  from IPython.display import SVG
  from keras.utils.vis_utils import model_to_dot
  
  #generate sin function with noise
  x = np.arange(0, 100, 0.1)
  noise = np.random.uniform(-0.1, 0.1, size=(1000,))
  Y = np.sin(x) + noise
  
  # Perform feature scaling
  scaler = MinMaxScaler()
  Y = scaler.fit_transform(Y.reshape(-1, 1))
  
  # split in train and test
  train_size = int(len(Y) * 0.7)
  test_size = len(Y) - train_size
  train, test = Y[0:train_size,:], Y[train_size:len(Y),:]
  
  def create_dataset(dataset, look_back=1):
      dataX, dataY = [], []
      for i in range(len(dataset)-look_back-1):
           a = dataset[i:(i+look_back), 0]
           dataX.append(a)
           dataY.append(dataset[i + look_back, 0])
      return np.array(dataX), np.array(dataY)
  
  # reshape into X=t and Y=t+1
  look_back = 10
  X_train, y_train = create_dataset(train, look_back)
  X_test, y_test = create_dataset(test, look_back)
  
  # LSTM network expects the input data in form of [samples, time steps, features]
  X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
  X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
  np.set_printoptions(threshold=np.inf)
  
  # compile model
  model = Sequential()
  model.add(LSTM(1, input_shape=(look_back, 1)))#, return_sequences=True))  &lt;== uncomment this
  model.compile(loss='mean_squared_error', optimizer='adam')
  SVG(model_to_dot(model).create(prog='dot', format='svg'))
  
  model.fit(X_train, y_train, validation_data=(X_test, y_test), 
  batch_size=10, epochs=10, verbose=2)
  prediction = model.predict(X_test, batch_size=1, verbose=0)
  prediction.reshape(-1) 
  #Transform back to original representation
  Y = scaler.inverse_transform(Y)
  prediction = scaler.inverse_transform(prediction)
  plt.plot(np.arange(0,Y.shape[0]), Y)
  plt.plot(np.arange(Y.shape[0] - X_test.shape[0] , Y.shape[0]), prediction, 'red')
  plt.show()
  error = mean_squared_error(y_test, prediction)
  print(error)
  </code></pre>