title: Error in Gradient Descent Calculation
body: <p>I tried to write a function to calculate gradient descent for a linear regression model. However the answers I was getting does not match the answers I get using the normal equation method. </p>
  
  <p>My sample data is:</p>
  
  <pre><code>df &lt;- data.frame(c(1,5,6),c(3,5,6),c(4,6,8))
  </code></pre>
  
  <p>with c(4,6,8) being the y values.</p>
  
  <pre><code>lm_gradient_descent &lt;- function(df,learning_rate, y_col=length(df),scale=TRUE){
  
  n_features &lt;- length(df) #n_features is the number of features in the data set
  
  #using mean normalization to scale features
  
  if(scale==TRUE){
  
  for (i in 1:(n_features)){
    df[,i] &lt;- (df[,i]-mean(df[,i]))/sd(df[,i])
      }
    }
    y_data &lt;- df[,y_col]
    df[,y_col] &lt;- NULL
    par &lt;- rep(1,n_features)
    df &lt;- merge(1,df)
    data_mat &lt;- data.matrix(df)
    #we need a temp_arr to store each iteration of parameter values so that we can do a 
    #simultaneous update
    temp_arr &lt;- rep(0,n_features)
    diff &lt;- 1
    while(diff&gt;0.0000001){
      for (i in 1:(n_features)){
        temp_arr[i] &lt;- par[i]-learning_rate*sum((data_mat%*%par-y_data)*df[,i])/length(y_data)
      }
      diff &lt;- par[1]-temp_arr[1]
      print(diff)
      par &lt;- temp_arr
    }
  
    return(par)
  }
  </code></pre>
  
  <p>Running this function,</p>
  
  <pre><code>lm_gradient_descent(df,0.0001,,0)
  </code></pre>
  
  <p>the results I got were </p>
  
  <pre><code>c(0.9165891,0.6115482,0.5652970)
  </code></pre>
  
  <p>when I use the normal equation method, I get </p>
  
  <pre><code>c(2,1,0).
  </code></pre>
  
  <p>Hope someone can shed some light on where I went wrong in this function.</p>