title: How to understand the tf.control_dependencies?
body: <p>There is a very weird example:</p>
  
  <pre><code>w = tf.get_variable("w", shape=(), dtype=tf.int32,
                      initializer=tf.constant_initializer(2))
  
  reset = tf.assign(w, 0)
  update = tf.assign(w, w + 3)
  update = tf.Print(update, [update])
  reset = tf.Print(reset, [reset])
  def body(i,x):
      with tf.control_dependencies([update]):
          t = tf.identity(w)
  
      with tf.control_dependencies([reset]):
          y = tf.identity(t)
      return i+1, y
  i, z = tf.while_loop(lambda i,z: i &lt; 20, body, [0,0])
  with tf.Session() as sess:
      tf.global_variables_initializer().run()
      print(sess.run(z))
  </code></pre>
  
  <p>The output is <code>5</code>. But how to explain it?
  As we can see that <code>reset</code> is not executed and <code>update</code> is executed only once
  utilizing <code>tf.Print</code>. But <code>tf.while_loop</code> will execute <code>body</code> 20 times. So it is amazing.</p>
  
  <h2>update</h2>
  
  <p>Another weird <a href="https://github.com/tensorflow/tensorflow/issues/4663#issuecomment-336504782" rel="nofollow noreferrer">example</a>:</p>
  
  <pre><code>import tensorflow as tf
  
  x = tf.Variable(0, dtype=tf.int32)
  
  old_val = tf.identity(x)
  with tf.control_dependencies([old_val]):
      new_val = tf.assign(x, x + 1)
  
  
  with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
  
      print(sess.run([old_val, new_val, x]))
  </code></pre>
  
  <p>The output is <code>[1,1,1]</code>. After looking the github issue, I am also confused.
  <code>sess.run()</code> is executed one by one, which means it runs <code>old_val</code>, then <code>new_val</code> and then <code>x</code>.</p>
  
  <p>When it runs <code>old_val</code>, it gets <code>0</code>, and when it runs <code>new_val</code>, it find <code>old_val</code> is dependencies of <code>new_val</code>, but <code>old_val</code> has been run. So it runs <code>new_val</code> immediately and gets <code>1</code>, and then runs <code>x</code>, gets <code>1</code>.</p>
  
  <p>So I think it should print <code>[0,1,1]</code>, so what's wrong? </p>