title: why does this Clojure code run out of memory?
body: <p>I have a twenty million line, sorted text file.  It has lots of duplicate lines.  I have some Clojure code that figures out how many instances there are of each unique line, i.e. the output is something like:</p>
  
  <pre><code>alpha 20
  beta 17
  gamma 3
  delta 4
  ...
  </code></pre>
  
  <p>The code works for smaller files, but on this larger one, it runs out of memory.  What am I doing wrong?  I assume that somewhere I am holding on to the head.</p>
  
  <pre><code>(require '[clojure.java.io :as io])
  
  (def bi-grams (line-seq (io/reader "the-big-input-file.txt")))
  
  (defn quick-process [input-list filename]
      (with-open [out (io/writer filename)] ;; e.g. "train/2gram-freq.txt"
          (binding [*out* out]
             (dorun (map (fn [[w v]] (println w "\t" (count v)))
                         (partition-by identity input-list)))
  
  (quick-process bi-grams "output.txt")
  </code></pre>