title: Finding duplicate columns in a data.table
body: <p>I have a pretty big data.table (500 x 2000), and I need to find out if any of the columns are duplicates, i.e., have the same values for all rows. Is there a way to efficiently do this within the data.table structure?</p>
  
  <p>I have tried a naive two loop approach with <code>all(col1 == col2)</code> for each pair of columns, but it takes too long. I have also tried converting it to a data.frame and using the above approach, and it still takes quite a long time.</p>
  
  <p>My current solution is to convert the data.table to a matrix and use the <code>apply()</code> function as:</p>
  
  <pre><code>similarity.matrix &lt;- apply(m, 2, function(x) colSums(x == m)))/nrow(m)
  </code></pre>
  
  <p>However, the approach forces the modes of all elements to be the same, and I'd rather not have that happen. What other options do I have?</p>
  
  <p>Here is a sample construction for the data.table:</p>
  
  <pre><code>m = matrix(sample(1:10, size=1000000, replace=TRUE), nrow=500, ncol=2000)
  DF = as.data.frame(m)
  DT = as.data.table(m)
  </code></pre>