title: tf.where causes optimiser to fail in tensorflow
body: <p>I want to check if I can solve <a href="http://koaning.io/switching-to-sampling-in-order-to-switch.html" rel="nofollow noreferrer">this</a> problem with tensorflow instead of pymc3. The experimental idea is that I am going to define a probibalistic system that contains a switchpoint. I can use sampling as a method of inference but I started wondering why I couldn't just do this with a gradient descent instead. </p>
  
  <p>I decided to do the gradient search in tensorflow but it seems like tensorflow is having a hard time performing a gradient search when <code>tf.where</code> is involved. </p>
  
  <p>You can find the code below. </p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  
  x1 = np.random.randn(50)+1
  x2 = np.random.randn(50)*2 + 5
  x_all = np.hstack([x1, x2])
  len_x = len(x_all)
  time_all = np.arange(1, len_x + 1)
  
  mu1 = tf.Variable(0, name="mu1", dtype=tf.float32)
  mu2 = tf.Variable(5, name = "mu2", dtype=tf.float32)
  sigma1 = tf.Variable(2, name = "sigma1", dtype=tf.float32)
  sigma2 = tf.Variable(2, name = "sigma2", dtype=tf.float32)
  tau = tf.Variable(10, name = "tau", dtype=tf.float32)
  
  mu = tf.where(time_all &lt; tau,
                tf.ones(shape=(len_x,), dtype=tf.float32) * mu1,
                tf.ones(shape=(len_x,), dtype=tf.float32) * mu2)
  sigma = tf.where(time_all &lt; tau,
                tf.ones(shape=(len_x,), dtype=tf.float32) * sigma1,
                tf.ones(shape=(len_x,), dtype=tf.float32) * sigma2)
  
  likelihood_arr = tf.log(tf.sqrt(1/(2*np.pi*tf.pow(sigma, 2)))) -tf.pow(x_all - mu, 2)/(2*tf.pow(sigma, 2))
  total_likelihood = tf.reduce_sum(likelihood_arr, name="total_likelihood")
  
  optimizer = tf.train.RMSPropOptimizer(0.01)
  opt_task = optimizer.minimize(-total_likelihood)
  init = tf.global_variables_initializer()
  
  with tf.Session() as sess:
      sess.run(init)
      print("these variables should be trainable: {}".format([_.name for _ in tf.trainable_variables()]))
      for step in range(10000):
          _lik, _ = sess.run([total_likelihood, opt_task])
          if step % 1000 == 0:
              variables = {_.name:_.eval() for _ in [mu1, mu2, sigma1, sigma2, tau]}
              print("step: {}, values: {}".format(str(step).zfill(4), variables))
  </code></pre>
  
  <p>You'll notice that the tau parameter does not change even though tensorflow seems to be aware of the variable and it's gradient. Any clue on what is going wrong? Is this something that can be calculated in tensorflow or do I need a different pattern? </p>