Question

I’m trying to solve Project Euler Problem 14 in a lazy way. Unfortunately, I may be trying to do the impossible: create a lazy sequence that is both lazy, yet also somehow ‘looks ahead’ for values it hasn’t computed yet.

The non-lazy version I wrote to test correctness was:

(defn chain-length [num]
  (loop [len 1
         n  num]
(cond
  (= n 1) len 
  (odd? n) (recur (inc len) (+ 1 (* 3 n)))
  true     (recur (inc len) (/ n 2)))))

Which works, but is really slow. Of course I could memoize that:

(def memoized-chain
   (memoize
     (fn [n]
       (cond
        (= n 1) 1 
         (odd? n) (+ 1 (memoized-chain (+ 1 (* 3 n))))
         true     (+ 1 (memoized-chain (/ n 2)))))))

However, what I really wanted to do was scratch my itch for understanding the limits of lazy sequences, and write a function like this:

(def lazy-chain
     (letfn [(chain [n] (lazy-seq
                         (cons (if (odd? n)
                                 (+ 1 (nth lazy-chain (dec (+ 1 (* 3 n)))))
                                 (+ 1 (nth lazy-chain (dec (/ n 2)))))
                               (chain (+ n 1)))))]
       (chain 1)))

Pulling elements from this will cause a stack overflow for n>2, which is understandable if you think about why it needs to look ‘into the future’ at n=3 to know the value of the tenth element in the lazy list because (+ 1 (* 3 n)) = 10.

Since lazy lists have much less overhead than memoization, I would like to know if this kind of thing is possible somehow via even more delayed evaluation or queuing?

Answer 1

The following gives me a lazy sequence of collatz values. On microbenchmarks on my machine, it mildly beats the memoized solution. On the downside, it also recurses too deeply to find 1 million collatz chain lengths, and the stack overflows somewhere between the 100,000th and 1,000,000th element, but that could be solved with a little work and recur.

 (defn collatz [n cache]
   (if-let [v (cache n)]
     [v cache]
     (let [[p cache] (if (odd? n)
                       (collatz (+ 1 (* 3 n)) cache)
                       (collatz (/ n 2) cache))]
       [(inc p) (assoc cache n (inc p))])))

 (def lazy-collatz
      (map first (iterate (fn [[v cache n]]
                            (concat (collatz n cache) [(inc n)]))
                          [1 {1 1} 2])))

Yet, it’s still not what I wanted: this same functionality without the hashmap. Thinking about Michal’s excellent comment and the concept of building a recursive tree, I guess what I wanted here was not a lazy sequence, but a lazy recursive datastructure of some sort, probably a tree. Do such dataflow techniques exist?

My original idea was to build chains of ‘delays’ from the unknown value (n) that continue to recurse until they hit a known number (like 1), and then unwind, populating the lazy datastructure with actual values as the recursion unwinds. If we think of a lazy sequence as being a lazy linked list, what I wanted was a lazy infinite length vector or tree that would find out its data dependencies automatically using the Collatz rule. A hashmap sufficed for this problem, but it is in some sense only an approximation of what I wanted: a lazy dataflow vector with a rule applied over how to populate the elements in the vector.

Extra Hard Challenge: Can anybody think of a way to easily represent such a lazy tree/vector without using a hashmap?