title: How does tensorflow batch_matmul work?
body: <p>Tensorflow has a function called <a href="https://www.tensorflow.org/versions/master/api_docs/python/math_ops.html#batch_matmul" rel="noreferrer">batch_matmul</a> which multiplies higher dimensional tensors. But I'm having a hard time understanding how it works, perhaps partially because I'm having a hard time visualizing it. </p>
  
  <p><a href="https://i.stack.imgur.com/xoiGQ.png" rel="noreferrer"><img src="https://i.stack.imgur.com/xoiGQ.png" alt="enter image description here"></a></p>
  
  <p>What I want to do is multiply a matrix by each slice of a 3D tensor, but I don't quite understand what the shape of tensor a is. Is z the innermost dimension? Which of the following is correct? </p>
  
  <p><a href="https://i.stack.imgur.com/sm1bZ.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/sm1bZ.jpg" alt="enter image description here"></a></p>
  
  <p>I would most prefer the first to be correct -- it's most intuitive to me and easy to see in the .eval() output. But I suspect the second is correct. </p>
  
  <p>Tensorflow says that batch_matmul performs:</p>
  
  <pre><code>out[..., :, :] = matrix(x[..., :, :]) * matrix(y[..., :, :])
  </code></pre>
  
  <p>What does that mean? What does that mean in the context of my example? What is being multiplied with with what? And why aren't I getting a 3D tensor the way I expected? </p>