title: Tensorflow gradient is always zero
body: <p>I have written a small Tensorflow program which convolves an image patch by the same convolution kernel <code>num_unrollings</code> times in a row, and then attempts to minimize the mean squared difference between the resulting values and a target output. </p>
  
  <p>However, when I run the model with <code>num_unrollings</code> greater than 1, the gradient of my my loss (<code>tf_loss</code>) term with respect to the convolution kernel (<code>tf_kernel</code>) is zero, so no learning occurs. </p>
  
  <p>Here is the smallest code (python 3) I can come up with which reproduces the problem, sorry about the length:</p>
  
  <pre class="lang-py prettyprint-override"><code>import tensorflow as tf
  import numpy as np
  
  batch_size = 1
  kernel_size = 3
  num_unrollings = 2
  
  input_image_size = (kernel_size//2 * num_unrollings)*2 + 1
  
  graph = tf.Graph()
  
  with graph.as_default():
      # Input data
      tf_input_images = tf.random_normal(
          [batch_size, input_image_size, input_image_size, 1]
      )
  
      tf_outputs = tf.random_normal(
          [batch_size]
      )
  
      # Convolution kernel
      tf_kernel = tf.Variable(
          tf.zeros([kernel_size, kernel_size, 1, 1])
      )
  
      # Perform convolution(s)
      _convolved_input = tf_input_images
      for _ in range(num_unrollings):
          _convolved_input = tf.nn.conv2d(
              _convolved_input, 
              tf_kernel, 
              [1, 1, 1, 1], 
              padding="VALID"
          )
  
      tf_prediction = tf.reshape(_convolved_input, shape=[batch_size])
  
      tf_loss = tf.reduce_mean(
          tf.squared_difference(
              tf_prediction,
              tf_outputs
          )
      )
  
      # FIXME: why is this gradient zero when num_unrollings &gt; 1??
      tf_gradient = tf.concat(0, tf.gradients(tf_loss, tf_kernel))
  
  # Calculate and report gradient
  with tf.Session(graph=graph) as session:
  
      tf.initialize_all_variables().run()
  
      gradient = session.run(tf_gradient)
  
      print(gradient.reshape(kernel_size**2))
      #prints [ 0.  0.  0.  0.  0.  0.  0.  0.  0.]
  </code></pre>
  
  <p>Thank you for your help!</p>