title: What is a projection layer in the context of neural networks?
body: <p>I am currently trying to understand the architecture behind the <em>word2vec</em> neural net learning algorithm, for representing words as vectors based on their context.</p>
  
  <p>After reading <a href="http://arxiv.org/pdf/1301.3781v3.pdf" rel="noreferrer">Tomas Mikolov paper</a> I came across what he defines as a <strong>projection layer</strong>. Even though this term is widely used when referred to <em>word2vec</em>, I couldn't find a precise definition of what it actually is in the neural net context.</p>
  
  <p><a href="https://i.stack.imgur.com/W46yb.png" rel="noreferrer"><img src="https://i.stack.imgur.com/W46yb.png" alt="Word2Vec Neural Net architecture"></a></p>
  
  <p>My question is, in the neural net context, what is a projection layer? Is it the name given to a hidden layer whose links to previous nodes share the same weights? Do its units actually have an activation function of some kind?</p>
  
  <p>￼Another resource that also refers more broadly to the problem can be found in <a href="http://www.coling-2014.org/COLING%202014%20Tutorial-fix%20-%20Tomas%20Mikolov.pdf" rel="noreferrer">this tutorial</a>, which also refers to a <em>projection layer</em> around page 67.</p>