title: How to optimize memory usage in dplyr + purrr
body: <p>I have an issue where after replicating data for a training and testing set, I'm showing a large amount of memory allocated to my user in Rstudio, but not being used in my R session. I've created a small example to reproduce my situation :)</p>
  
  <p>This code runs a bunch of model, based on different formulas, algorithms, and parameter sets that I give it. It is a function, but I've created a simple script for reprex.</p>
  
  <pre><code>library(dplyr)
  library(purrr)
  library(modelr)
  library(tidyr)
  library(pryr)
  
  # set my inputs
  data &lt;- mtcars
  formulas &lt;- c(test1 = mpg ~ cyl + wt + hp,
                test2 = mpg ~ cyl + wt)
  params = list()
  methods &lt;- "lm"
  
  n &lt;- 20 # num of cv splits
  mult &lt;- 10 # number of times I want to replicate some of the data
  frac &lt;- .25 # how much I want to cut down other data (fractional)
  
  ### the next few chunks get the unique combos of the inputs.
  if (length(params) != 0) {
    cross_params &lt;- params %&gt;% 
      map(cross) %&gt;% 
      map_df(enframe, name = "param_set", .id = "method") %&gt;% 
      list
  } else cross_params &lt;- NULL
  
  methods_df &lt;- tibble(method = methods) %&gt;% 
    list %&gt;% 
    append(cross_params)  %&gt;% 
    reduce(left_join, by = "method") %&gt;% 
    split(1:nrow(.))
  
  # wrangle formulas into a split dataframe
  formulas_df &lt;- tibble(formula = formulas,
                        name = names(formulas)) %&gt;% 
    split(.$name)
  
  # split out the data into n random train-test combos
  cv_data &lt;- data %&gt;% 
    crossv_kfold(n) %&gt;% # rsample?
    mutate_at(vars(train:test), ~map(.x, as_tibble))
  
  # sample out if needed
  cv_data_samp &lt;- cv_data %&gt;%
    mutate(train = modify(train, 
                          ~ .x %&gt;% 
                            split(.$gear == 4) %&gt;% 
                            # take a sample of the non-vo data
                            modify_at("FALSE", sample_frac, frac) %&gt;% 
                            # multiply out the vo-on data
                            modify_at("TRUE", function(.df) {
                              map_df(seq_along(1:mult), ~ .df) 
                            }) %&gt;% 
                            bind_rows))
  
  # get all unique combos of formula and method
  model_combos &lt;- list(cv = list(cv_data_samp), 
                       form = formulas_df, 
                       meth = methods_df) %&gt;% 
    cross %&gt;%
    map_df(~ bind_cols(nest(.x$cv), .x$form, .x$meth)) %&gt;% 
    unnest(data, .preserve = matches("formula|param|value")) %&gt;% 
    {if ("value" %in% names(.)) . else mutate(., value = list(NULL))} 
  
  # run the models
  model_combos %&gt;% 
    # put all arguments into a single params column
    mutate(params = pmap(list(formula = formula, data = train), list)) %&gt;%
    mutate(params = map2(params, value, ~ append(.x, .y))) %&gt;%
    mutate(params = modify(params, discard, is.null)) %&gt;%
    # run the models
    mutate(model = invoke_map(method, params))  
  
  mem_change(rm(data, cv_data, cv_data_samp))
  mem_used()
  </code></pre>
  
  <p>Now after I do this, my <code>mem_used</code> comes out to 77.3mb, but I'm seeing roughly double that (160Mb) allocated to my R user. This really explodes when my data is 3 Gb, which is my real-life case. I end up using 100Gb and tying up a whole server :(. </p>
  
  <p>What is going on and how can I optimize?</p>
  
  <p>Any help appreciated!!!</p>