title: Keras LSTM Autoencoder time-series reconstruction
body: <p>I am trying to reconstruct time series data with LSTM Autoencoder (Keras).
  Now I want train autoencoder on small amount of samples (5 samples, every sample is 500 time-steps long and have 1 dimension). I want to make sure that model can reconstruct that 5 samples and after that I will use all data (6000 samples).</p>
  
  <pre><code>window_size = 500
  features = 1
  data = data.reshape(5, window_size, features)
  
  model = Sequential()
  
  model.add(LSTM(256, input_shape=(window_size, features), 
  return_sequences=True))
  model.add(LSTM(128, input_shape=(window_size, features), 
  return_sequences=False))
  model.add(RepeatVector(window_size))
  
  model.add(LSTM(128, input_shape=(window_size, features), 
  return_sequences=True))
  model.add(LSTM(256, input_shape=(window_size, features), 
  return_sequences=True))
  model.add(TimeDistributed(Dense(1)))
  
  model.compile(optimizer='adam', loss='mse')
  model.fit(data, data, epochs=100, verbose=1)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/myTSa.png" rel="noreferrer">Model</a></p>
  
  <p>Training:</p>
  
  <pre><code>Epoch 1/100
  5/5 [==============================] - 2s 384ms/step - loss: 0.1603
  ...
  Epoch 100/100
  5/5 [==============================] - 2s 388ms/step - loss: 0.0018
  </code></pre>
  
  <p>After training, I tried reconstruct one of 5 samples:</p>
  
  <pre><code>yhat = model.predict(np.expand_dims(data[1,:,:], axis=0), verbose=0)
  </code></pre>
  
  <p>Reconstitution: Blue<br>
  Input: Orange   </p>
  
  <p><img src="https://i.stack.imgur.com/Tzsdg.png" alt="Reconstion (blue) vs Input (orange)"></p>
  
  <p>Why is reconstruction so bad when loss is small? How can I make model better? Thanks.</p>