title: Keras: Suprising compare simple neural network 2-output and 1-output
body: <p>I wrote two simple models to predict linear data. First, produce 1-output and work correctly in opposite to second which produces 2-output (but I use only first output). </p>
  
  <pre><code>model1 = Sequential([
      Dense(1, input_shape=(1,))
  ])
  
  model2 = Sequential([
      Dense(2, input_shape=(1,))
  ])
  </code></pre>
  
  <p>I use standard mse in both cases.</p>
  
  <pre><code>def mse1(y_true, y_pred):
      return K.mean(K.square(y_pred - y_true), axis=-1)
  
  def mse2(y_true, y_pred): 
      return K.mean(K.square(y_pred[:,0] - y_true), axis=-1)
  
  model1.compile(optimizer='adam', loss=mse1)
  model2.compile(optimizer='adam', loss=mse2)
  
  model1.fit(np.asarray(range(len(data)),dtype=np.float32), np.asarray(data), epochs=10000, batch_size=100)
  model2.fit(np.asarray(range(len(data)),dtype=np.float32), np.asarray(data), epochs=10000, batch_size=100)
  
  out1 = model1.predict(np.asarray(range(len(data))))
  out2 = model2.predict(np.asarray(range(len(data))))[:,0]
  
  plt.scatter(range(len(l)), data,  color='r')
  plt.scatter(range(len(l)), out1, color='b')
  plt.scatter(range(len(l)), out2, color='g') 
  plt.show()
  </code></pre>
  
  <p>I try to repeat this experiment several times and always get the same result - blue (model1) work correctly but green (model2) are below most of the data points.
  <a href="https://i.stack.imgur.com/6VE5m.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/6VE5m.png" alt="model1 vs model2"></a></p>