title: Multiple issues with axes while implementing a Seq2Seq with attention in CNTK
body: <p>I'm trying to implement a Seq2Seq model with attention in CNTK, something very similar to <a href="https://github.com/Microsoft/CNTK/blob/v2.1/Tutorials/CNTK_204_Sequence_To_Sequence.ipynb" rel="nofollow noreferrer">CNTK Tutorial 204</a>. However, several small differences lead to various issues and error messages, which I don't understand. There are many questions here, which are probably interconnected and all stem from some single thing I don't understand.</p>
  
  <p>Note (in case it's important). My input data comes from <code>MinibatchSourceFromData</code>, created from NumPy arrays that fit in RAM, I don't store it in a CTF.</p>
  
  <pre><code>ins = C.sequence.input_variable(input_dim, name="in", sequence_axis=inAxis)
  y = C.sequence.input_variable(label_dim, name="y", sequence_axis=outAxis)
  </code></pre>
  
  <p>Thus, the shapes are <code>[#, *](input_dim)</code> and <code>[#, *](label_dim)</code>.</p>
  
  <p><strong>Question 1:</strong> When I run the <a href="https://github.com/Microsoft/CNTK/blob/v2.1/Tutorials/CNTK_204_Sequence_To_Sequence.ipynb" rel="nofollow noreferrer">CNTK 204 Tutorial</a> and dump its graph into a <code>.dot</code> file using <code>cntk.logging.plot</code>, I see that its input shapes are <code>[#](-2,)</code>. How is this possible?</p>
  
  <ul>
  <li>Where did the sequence axis (<code>*</code>) disappear?</li>
  <li>How can a dimension be negative?</li>
  </ul>
  
  <p><strong>Question 2:</strong> In the same tutorial, we have <code>attention_axis = -3</code>. I don't understand this. In my model there are 2 dynamic axis and 1 static, so "third to last" axis would be <code>#</code>, the batch axis. But attention definitely shouldn't be computed over the batch axis.<br>
  I hoped that looking at the actual axes in the tutorial code would help me understand this, but the <code>[#](-2,)</code> issue above made this even more confusing.</p>
  
  <p>Setting <code>attention_axis</code> to <code>-2</code> gives the following error:</p>
  
  <pre><code>RuntimeError: Times: The left operand 'Placeholder('stab_result', [#, outAxis], [128])'
                rank (1) must be &gt;= #axes (2) being reduced over.
  </code></pre>
  
  <p>during creation of the training-time model:
  </p>
  
  <pre><code>def train_model(m):
      @C.Function
      def model(ins: InputSequence[Tensor[input_dim]],                  
                labels: OutputSequence[Tensor[label_dim]]):
          past_labels = Delay(initial_state=C.Constant(seq_start_encoding))(labels)
          return m(ins, past_labels)  #&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt; HERE
      return model
  </code></pre>
  
  <p>where <code>stab_result</code> is a <code>Stabilizer</code> right before the final <code>Dense</code> layer in the decoder. I can see in the dot-file that there are spurious trailing dimensions of size 1 that appear in the middle of the <code>AttentionModel</code> implementation.</p>
  
  <p>Setting <code>attention_axis</code> to <code>-1</code> gives the following error:</p>
  
  <pre><code>RuntimeError: Binary elementwise operation ElementTimes: Left operand 'Output('Block346442_Output_0', [#, outAxis], [64])'
                shape '[64]' is not compatible with right operand 
                'Output('attention_weights', [#, outAxis], [200])' shape '[200]'.
  </code></pre>
  
  <p>where 64 is my <code>attention_dim</code> and 200 is my <code>attention_span</code>. As I understand, the elementwise <code>*</code> inside the attention model definitely shouldn't be conflating these two together, therefore <code>-1</code> is definitely not the right axis here.</p>
  
  <p><strong>Question 3:</strong> Is my understanding above correct? What should be the right axis and why is it causing one of the two exceptions above?</p>
  
  <p>Thanks for the explanations!</p>