Question

Not sure what is the behaviour I observe while using flatten when constructing a lazy sequence.

Looking at the source in clojure.core I can see that the flatten function makes a call to filter and hence should return a lazy sequence – I think. And yet the following snippet gives me a stackoverflow error. In the snippet when the call to flatten is replaced with a call to concat, it works just fine

(defn l-f [c]
  (if (nil? c) []
    (lazy-seq  (flatten (cons  [[ :h :j] :a :B] (l-f (rest c))))))) 


    (take 10 (l-f (repeat 2))) is how I invoke it.

This is a rather contrived example. Also I am aware that flatten and concat will give me sequences where the nesting levels are different.

I am trying to figure out why flatten seems to break the laziness, even though my (limited) understanding of the code in clojure.core suggests otherwise.

Answer 1

Laziness only takes you so far – laziness just means that the sequence isn’t fully realized at the time that it’s created, but building one lazy sequence from another sometimes involves looking ahead a few values. In this case, the implementation of flatten doesn’t play nicely with the recursive way in which you’re calling it.

First, the flatten function calls tree-seq to do a depth-first traversal of the contents of the collection. In turn, tree-seq calls mapcat with the provided sequence, which delegates to apply, which realizes the first few items in the sequence to determine the arity of the function to invoke. Realizing the first few items in the sequence causes a recursive call to l-f, which calls flatten on the remaining arguments, and gets stuck in an infinite loop.

In this particular situation, there’s no need to call flatten recursively, because any call after the first will have no effect. So your function can be fixed by separating out the generation of the lazy sequence from the flattening of it:

(defn l-f [c]                                                      
  (letfn [(l-f-seq [x] (if-let [s (seq x)]                         
                         (lazy-seq (cons [[:h :j] :a :B] (l-f-seq (rest s))))
                         []))]                                               
    (flatten (l-f-seq c))))