title: How to allocate/append a large column of Date objects to a data-frame
body: <p>I have a data-frame (3 cols, 12146637 rows) called <strong>tr.sql</strong> which occupies 184Mb.
  (it's backed by SQL, it is the contents of my dataset which I read in via read.csv.sql)</p>
  
  <p>Column 2 is <strong>tr.sql$visit_date</strong>. SQL does not allow natively representing dates as an R Date object, this is important for how I need to process the data.</p>
  
  <p>Hence I want to copy the contents of <strong>tr.sql</strong> to a new data-frame <strong>tr</strong>
  (where the visit_date column can be natively represented as Date (chron::Date?). Trust me, this makes exploratory data analysis easier, for now this is how I want to do it - I might use native SQL eventually but please don't quibble that for now.)</p>
  
  <hr>
  
  <p>Here is my solution (thanks to gsk and everyone) + workaround:</p>
  
  <pre><code>tr &lt;- data.frame(customer_id=integer(N), visit_date=integer(N), visit_spend=numeric(N))
  # fix up col2's class to be Date
  class(tr[,2]) &lt;- 'Date'
  </code></pre>
  
  <p>then workaround copying tr.sql -> tr in chunks of (say) N/8 using a for-loop, so that the temporary involved in the str->Date conversion does not out-of-memory, and a garbage-collect after each:</p>
  
  <p>for (i in 0:7) {
      from &lt;- floor(i*N/8)
      to &lt;- floor((i+1)*N/8) -1
      if (i==7)
          to &lt;- N
      print(c("Copying tr.sql$visit_date",from,to," ..."))
      tr$visit_date[from:to] &lt;- as.Date(tr.sql$visit_date[from:to])
      gc()
  }
  rm(tr.sql)
  memsize_gc() ... # only 321 Mb in the end! (was ~1Gb during copying)</p>
  
  <hr>
  
  <p><strong>The problem is allocating then copying the visit_date column.</strong>
  Here is the dataset and code, I am having multiple separate problems with this, explanation below: </p>
  
  <pre><code>'training.csv' looks like...
  customer_id,visit_date,visit_spend 
  2,2010-04-01,5.97 
  2,2010-04-06,12.71 
  2,2010-04-07,34.52 
  </code></pre>
  
  <p>and code:</p>
  
  <pre><code># Read in as SQL (for memory-efficiency)...
  library(sqldf)
  tr.sql &lt;- read.csv.sql('training.csv')
  gc()
  memory.size()
  
  # Count of how many rows we are about to declare
  N &lt;- nrow(tr.sql)
  # Declare a new empty data-frame with same columns as the source d.f.
  # Attempt to declare N Date objects (fails due to bad qualified name for Date)
  # ... does this allocate N objects the same as data.frame(colname = numeric(N)) ?
  tr &lt;- data.frame(visit_date = Date(N))
  tr &lt;- tr.sql[0,]
  # Attempt to assign the column - fails
  tr$visit_date &lt;- as.Date(tr.sql$visit_date)
  # Attempt to append (fails)
  &gt; tr$visit_date &lt;- append(tr$visit_date, as.Date(tr.sql$visit_date))
  Error in `$&lt;-.data.frame`(`*tmp*`, "visit_date", value = c("14700", "14705",  : 
    replacement has 12146637 rows, data has 0
  </code></pre>
  
  <ol>
  <li>The second line that tries to declare <code>data.frame(visit_date = Date(N))</code> fails, I don't know the correct qualified name with namespace for Date object (tried <code>chron::Date , Dates::Date</code>? don't work)</li>
  <li>Both the attempt to assign and append fail. Not even sure whether it is legal, or efficient, to use append on a single large column of a data-frame.</li>
  </ol>
  
  <p>Remember these objects are big, so avoid using temporaries.
  Thanks in advance... </p>