Question

(take 2 (for [x (range 10)
              :let [_ (println x)]
              :when (even? x)] x))
>> (* 0
* 1
* 2
* 3
* 4
* 5
* 6
* 7
* 8
* 9
0 2)

I assumed I was just being remarkably dense. But no, it turns out that Clojure actually evaluates the first 32 elements of any lazy sequence (if available). Ouch.

I had a for with a recursive call in the :let. I was very curious as to why computation seemed to be proceeding in a breadth first rather than depth first fashion. It seems that computation (although, to be fair, not memory) was exploding as I kept going down all the upper branches of the recursive tree. Clojure’s 32-chunking was forcing breadth first evaluation, even though the logical intent of the code was depth first.

Anyway, is there any simple way to force 1-chunking rather than 32-chunking of lazy sequences?

Answer 1

Michaes Fogus has written a blog entry on disabling this behavior by providing a custom ISeq implementation.

To steal shamelessly from the modified version by Colin Jones:

(defn seq1 [#^clojure.lang.ISeq s]
  (reify clojure.lang.ISeq
    (first [_] (.first s))
    (more [_] (seq1 (.more s)))
    (next [_] (let [sn (.next s)] (and sn (seq1 sn))))
    (seq [_] (let [ss (.seq s)] (and ss (seq1 ss))))
    (count [_] (.count s))
    (cons [_ o] (.cons s o))
    (empty [_] (.empty s))
    (equiv [_ o] (.equiv s o))))

A simpler approach is given in The Joy of Clojure:

(defn seq1 [s]
  (lazy-seq
    (when-let [[x] (seq s)]
       (cons x (seq1 (rest s))))))