title: Dropout layer before or after LSTM. What is the difference?
body: <p>Suppose that we have an LSTM model for time series forecasting. Also, this is a multivariate case, so we're using more than one feature for training the model.</p>
  
  <pre><code>ipt   = Input(shape = (shape[0], shape[1])
  x     = Dropout(0.3)(ipt) ## Dropout before LSTM.
  x     = CuDNNLSTM(10, return_sequences = False)(x)
  out   = Dense(1, activation='relu')(x)
  </code></pre>
  
  <p>We can add <code>Dropout</code> layer before LSTM (like the above code) or after LSTM.</p>
  
  <ul>
  <li><p>If we add it before LSTM, is it applying dropout on timesteps (different lags of time series), or different input features, or both of them?</p></li>
  <li><p>If we add it after LSTM and because <code>return_sequences</code> is <code>False</code>, what is dropout doing here?</p></li>
  <li><p>Is there any different between <code>dropout</code> option in <code>LSTM</code> and dropout layer before <code>LSTM</code> layer?</p></li>
  </ul>