title: How to vectorize equations?
body: <p>I'm trying to implement the Softmax regression algorithm to solve the K-classifier problem after watching Professor Andrew Ng's lectures on GLM. I <em>thought</em> I understood everything he was saying until it finally came to writing the code to implement the cost function for Softmax regression, which is as follows:</p>
  
  <p><img src="https://i.stack.imgur.com/ryPJk.png" alt="Cost function for Softmax Regression with Weight Decay"></p>
  
  <p>The problem I am having is trying to figure out a way to vectorize this. Again I <em>thought</em> I understood how to go about vectorizing equations like this since I was able to do it for linear and logistic regression, but after looking at that formula I am stuck.</p>
  
  <p>While I would love to figure out a vectorized solution for this (I realize there is a similar question posted already: <a href="https://stackoverflow.com/questions/8998321/vectorized-implementation-of-softmax-regression">Vectorized Implementation of Softmax Regression</a>), what I am more interested in is whether any of you can tell me a way (your way) to <em>methodically</em> convert equations like this into vectorized forms. For example, for those of you who are experts or seasoned veterans in ML, when you read of new algorithms in the literature for the first time, and see them written in similar notation to the equation above, how do you go about converting them to vectorized forms? </p>
  
  <p>I realize I might be coming off as being like the student who is asking Mozart, "How do you play the piano so well?" But my question is simply motivated from a desire to become better at this material, and assuming that not everyone was born knowing how to vectorize equations, and so someone out there must have devised their own system, and if so, please share! Many thanks in advance!</p>
  
  <p>Cheers </p>