title: tensorflow einsum vs. matmul vs. tensordot
body: <p>In tensorflow, the functions <code>tf.einsum</code>, <code>tf.matmul</code>, and <code>tf.tensordot</code> can all be used for the same tasks. (I realize that <code>tf.einsum</code> and <code>tf.tensordot</code> have more general definitions; I also realize that <code>tf.matmul</code> has batch functionality.) In a situation where any of the three could be used, does one function tend to be fastest? Are there other recommendation rules?</p>
  <p>For example, suppose that <code>A</code> is a rank-2 tensor, and <code>b</code> is rank-1 tensor, and you want to compute the product <code>c_j = A_ij b_j</code>. Of the three options:</p>
  <p><code>c = tf.einsum('ij,j-&gt;i', A, b)</code></p>
  <p><code>c = tf.matmul(A, tf.expand_dims(b,1))</code></p>
  <p><code>c = tf.tensordot(A, b, 1)</code></p>
  <p>Is any generally preferable to the others?</p>