title: Lazy stateful processing with sequence libraries
body: <p>Let's say I want to process a sequence, where the processing is stateful, and I want to do this in a lazy way using the Clojure sequence libraries.</p>
  
  <p>As a concrete example, assume I want to implement <code>distinct</code>, which is naturally implemented as a stateful filter, keeping track of the elements seen. My first stab is to make no use of sequence libraries, but instead <code>lazy-seq</code>:</p>
  
  <pre><code>(defn distinct' [coll]
    (let [process (fn process [coll seen]
                    (lazy-seq
                      (when-let [[x &amp; r] (seq coll)]
                        (if (contains? seen x)
                          (process r seen)
                          (cons x (process r (conj seen x)))))))]
      (process coll #{})))
  </code></pre>
  
  <p>I'm OK with the above, but, I'd prefer to instead make use of methods like <code>map</code> and <code>filter</code> if I can. I'm struggling with how to do this cleanly. One approach that works is to make use of <code>reductions</code>:</p>
  
  <pre><code>(defn distinct'' [coll]
    (-&gt;&gt; (reductions (fn [[_ _ seen] x]
                       (if (contains? seen x)
                         [false nil seen]
                         [true x (conj seen x)]))
                     [false nil #{}]
                     coll)
         (filter first)
         (map second)))
  </code></pre>
  
  <p>Fundamentally, <code>distinct''</code> fulfills the goal of using higher-level abstractions (<code>map</code>, <code>filter</code>, <code>reductions</code>) while remaining lazy. But it fails in being overly complex, especially with the vector I have it passing along as the accumulator.</p>
  
  <p>If I try to directly embed state in a predicate to be used with <code>filter</code>, while it is closer to what I'm imagining, it just seems to be "wrong", and I'm embarrassed to even write the following code (<code>filter</code>'s docs even say that the predicate should be free of side effects):</p>
  
  <pre><code>(defn distinct''' [coll]
    (let [seen (atom #{})]
      (filter (fn [x]
                (if (contains? @seen x)
                  false
                  (do (swap! seen conj x)
                      true)))
              coll)))
  </code></pre>
  
  <p>My question: </p>
  
  <p>Is there a way to do lazy stateful processing like this with sequence libraries in a clean way (even for this <code>distinct</code> example)? Or, is <code>lazy-seq</code> often the cleanest approach?</p>