title: How can I speed up the training of my random forest?
body: <p>I'm trying to train several random forests (for regression) to have them compete and see which feature selection and which parameters give the best model.</p>
  
  <p>However the trainings seem to take an insane amount of time, and I'm wondering if I'm doing something wrong.</p>
  
  <p>The dataset I'm using for training (called <code>train</code> below) has 217k lines, and 58 columns (of which only 21 serve as predictors in the random forest. They're all <code>numeric</code> or <code>integer</code>, with the exception of a boolean one, which is of class <code>character</code>. The <code>y</code> output is <code>numeric</code>). </p>
  
  <p>I ran the following code four times, giving the values <code>4</code>, <code>100</code>, <code>500</code>, <code>2000</code> to <code>nb_trees</code> :</p>
  
  <pre><code>library("randomForest")
  nb_trees &lt;- #this changes with each test, see above
  ptm &lt;- proc.time()
  fit &lt;- randomForest(y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 
      + x10 + x11 + x12 + x13 + x14 + x15 + x16 + x17 + x18 + x19 
      + x20 + x21, 
      data = train, 
      ntree = nb_trees, 
      do.trace=TRUE)
  proc.time() - ptm
  </code></pre>
  
  <p>Here is how long each of them took to train :</p>
  
  <pre><code>nb_trees | time
  4          4mn
  100        1h 41mn
  500        8h 40mn
  2000       34h 26mn
  </code></pre>
  
  <p>As my company's server has 12 cores and 125Go of RAM, I figured I could try to parallelize the training, following <a href="https://stackoverflow.com/a/7831848/4348534">this answer</a> (however, I used the <code>doParallel</code> package because it seemed to be running forever with <code>doSNOW</code>, I don't know why. And I can't find where I saw that <code>doParallel</code> would work too, sorry).</p>
  
  <pre><code>library("randomForest")
  library("foreach")
  library("doParallel")
  nb_trees &lt;- #this changes with each test, see table below
  nb_cores &lt;- #this changes with each test, see table below
  cl &lt;- makeCluster(nb_cores)
  registerDoParallel(cl)
  ptm &lt;- proc.time()
  fit &lt;- foreach(ntree = rep(nb_trees, nb_cores), .combine = combine, .packages = "randomForest") 
      %dopar% {
          randomForest(y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 
          + x10 + x11 + x12 + x13 + x14 + x15 + x16 + x17 + x18 + x19 
          + x20 + x21,
          data = train, 
          ntree = ntree,
          do.trace=TRUE)}
  proc.time() - ptm
  stopCluster(cl)
  </code></pre>
  
  <p>When I run it, it takes a shorter time than non-parallelized code :</p>
  
  <pre><code>nb_trees | nb_cores | total number of trees              | time
  1          4          4                                    2mn13s
  10         10         100                                  52mn
  9          12         108 (closest to 100 with 12 cores)   59mn
  42         12         504 (closest to 500 with 12 cores)   I won't be running this one
  167        12         2004 (closest to 2000 with 12 cores) I'll run it next week-end
  </code></pre>
  
  <p>However, I think it's still taking a lot of time, isn't it ? I'm aware it takes time to combine the trees into the final forest, so I didn't expect it to be 12 times faster with 12 cores, but it's only ~2 times faster... </p>
  
  <ul>
  <li>Is this normal ? </li>
  <li>If it isn't, is there anything I can do with my data and/or my code to radically decrease the running time ? </li>
  <li>If not, should I tell the guy in charge of the server that it should be much faster ?</li>
  </ul>
  
  <p>Thanks for your answers.</p>
  
  <p>Notes :</p>
  
  <ul>
  <li>I'm the only one using this server</li>
  <li>for my next tests, I'll get rid of the columns that are not used in the random forest</li>
  <li>I realized quite late that I could improve the running time by calling <code>randomForest(predictors,decision)</code> instead of <code>randomForest(decision~.,data=input)</code>, and I'll be doing it from now on, but I think my questions above still holds.</li>
  </ul>