title: Is there a _fast_ way to run a rolling regression inside data.table?
body: <p>I am running rolling regressions in R, using with the data stored in a <code>data.table</code>. </p>
  
  <p>I have a working version, however it <em>feels</em> like a hack -- I am really using what i know from the <code>zoo</code> package, and none of the magic in <code>data.table</code> ... thus, it feels slower than it ought to be. </p>
  
  <p>Incorporating Joshua's suggestion - below - there is a speedup of ~12x by using <code>lm.fit</code> rather than <code>lm</code>. </p>
  
  <p>(revised) Example code: </p>
  
  <pre><code>require(zoo)
  require(data.table)
  require(rbenchmark)
  set.seed(1)
  
  tt &lt;- seq(as.Date("2011-01-01"), as.Date("2012-01-01"), by="day")
  px &lt;- rnorm(366, 95, 1)
  
  DT &lt;- data.table(period=tt, pvec=px)
  
  dtt &lt;- DT[,tnum:=as.numeric(period)][, list(pvec, tnum)]
  dtx &lt;- as.matrix(DT[,tnum:=as.numeric(period)][, tnum2:= tnum^2][, int:=1][, list(pvec, int, tnum, tnum2)])
  
  rollreg &lt;- function(dd) coef(lm(pvec ~ tnum + I(tnum^2), data=as.data.frame(dd)))
  rollreg.fit &lt;- function(dd) coef(lm.fit(y=dd[,1], x=dd[,-1]))
  
  rr &lt;- function(dd) rollapplyr(dd, width=20, FUN = rollreg, by.column=FALSE)
  rr.fit &lt;- function(dd) rollapplyr(dd, width=20, FUN = rollreg.fit, by.column=FALSE)
  
  bmk &lt;- benchmark(rr(dtt), rr.fit(dtx), 
           columns = c('test', 'elapsed', 'relative'),
           replications = 10,
           order = 'elapsed'
         )
  
       test elapsed relative
  2 rr.fit(dtx)    0.48   1.0000
  1     rr(dtt)    5.85  12.1875
  </code></pre>
  
  <p><em>Trying</em> to apply the knowledge displayed <a href="https://stackoverflow.com/questions/12150472/using-data-table-to-speed-up-rollapply">here</a> and <a href="https://stackoverflow.com/questions/11676476/r-data-table-sliding-window">here</a>, I cooked up the following simple rolling regression function that <em>I think</em> uses some of the speed of data.table operations. </p>
  
  <p>Note that the problem is a little different (and more realistic): take a vector, add lags, and regress on itself.  This class of AR-type problems is pretty broad. </p>
  
  <p>I am sharing it here as it may be of use, and i'm <strong>sure</strong> that it can be improved (i'll update as I improve): </p>
  
  <pre><code>require(data.table)
  set.seed(1)
  x  &lt;- rnorm(1000)
  DT &lt;- data.table(x)
  DTin &lt;- data.table(x)
  
  lagDT &lt;- function(DTin, varname, l=5)
  {
      i = 0
      while ( i &lt; l){
          expr &lt;- parse(text = 
                    paste0(varname, '_L', (i+1), 
                       ':= c(rep(NA, (1+i)),', varname, '[-((length(',     varname, ') - i):length(', varname, '))])'
                   )
                )
      DTin[, eval(expr)]
      i &lt;- i + 1
  }
  return(DTin)
  }   
  
  rollRegDT &lt;- function(DTin, varname, k=20, l=5)
  {
  adj &lt;- k + l - 1
  .x &lt;- 1:(nrow(DTin)-adj)
  DTin[, int:=1]
  dtReg &lt;- function(dd) coef(lm.fit(y=dd[-c(1:l),1], x=dd[-c(1:l),-1]))
  eleNum &lt;- nrow(DTin)*(l+1)
  outMatx &lt;- matrix(rep(NA, eleNum), ncol = (l+1))
  colnames(outMatx) &lt;- c('intercept', 'L1', 'L2', 'L3', 'L4', 'L5')
  for (i in .x){
      dt_m &lt;- as.matrix(lagDT(DTin[i:(i+adj), ], varname, l))
      outMatx[(i+(adj)),] &lt;- dtReg(dt_m)
  }
  return(outMatx)
  }
  
  rollCoef &lt;- rollRegDT(DT, varname='x')
  </code></pre>