Question

I’m having trouble downloading images using Clojure, there seems to be an issue with the way the following code works: –

(defn download-image [url filename]
    (->> (slurp url) (spit filename)))

This will ‘download’ the file to the location I specify but the file is unreadable by any image application I try to open it with (for example, attempting to open it in a web browser just return a blank page, attempting to open it in Preview (osx) says it’s a corrupted file)

I’m thinking this is might be because slurp should only really be used for text files rather than binary files

Could anyone point me in the right direction for my code to work properly? Any help would be greatly appreciated!

Answer 1

slurp uses java.io.Reader underneath, which will convert the representation to a string, and this is typically not compatible with binary data. Look for examples that use input-stream instead. In some ways, this can be better, because you can transfer the image from the input buffer to the output buffer without having to read the entire thing into memory.

edit

Since people seem to find this question once in awhile and I needed to rewrite this code again. I thought I’d add an example. Note, this does not stream the data, it collects it into memory and returns it an array of bytes.

(require '[clojure.java.io :as io])

(defn blurp [f]
  (let [dest (java.io.ByteArrayOutputStream.)]
    (with-open [src (io/input-stream f)]
      (io/copy src dest))
    (.toByteArray dest)))

Test…

(use 'clojure.test)

(deftest blurp-test
  (testing "basic operation"
    (let [src (java.io.ByteArrayInputStream. (.getBytes "foo" "utf-8"))]
      (is (= "foo" (-> (blurp src) (String. "utf-8")))))))

Example…

user=> (blurp "http://www.lisperati.com/lisplogo_256.png")
#<byte[] [B@15671adf>