Question

When I run a 221 line .csv file — parsed with clojure-csv — into this function

(defn test-key-inclusion
    "Accepts csv-data param and an index, a second csv-data param and an index,
     and searches the second csv-data instances' rows (at index) to see if
     the first file's data is located in the second csv-data instance."

    [csv-data1 pkey-idx1 csv-data2 pkey-idx2 lnam-idx fnam-idx]

    (reduce
        (fn [out-log csv-row1]
            (let [cmp-val (nth csv-row1 pkey-idx1 nil)
                  lnam (nth csv-row1 lnam-idx nil)
                  fnam (nth csv-row1 fnam-idx)
                  temp-rc (first (key-pres? cmp-val pkey-idx2 csv-data2))]

            (concat out-log (sorted-map cmp-val (vector lnam fnam)))))
         {}
         csv-data1))

and then print the result, everything’s fine.

If I run a 2672 line .csv file — also parsed with clojure-csv — through the function above and then try to print it, I get a stack overflow error — Exception in thread “main” java.lang.StackOverflowError

So my questions are:

1) Should wrapping the call to this function inside lazy-seq cure my problem?

2) I don’t want a list, so will wrapping the lazy-seq call inside a vec turn
my sequence back into a vector without realizing the whole sequence in memory, that is make the lazy-seq un-lazy again?

Thank you.

Answer 1

1) i expect that making the sequence lazy not to help because print will evaluate realize it before printing it. instead try doseq or (map print my-seq) to print it in smaller chunks.

2) yes wrapping it in vec will give you what you want 🙂 though wrapping your reduce with an into would keep it a vector the whole time. ie: (reduce into [] [[1] [2] [3]] ) –> [1 2 3]

  (into out-log (sorted-map cmp-val (vector lnam fnam)))))