Question

I have a lein project (using cascalog–but that’s not particularly important). I’m trying to externalize properties like paths to files, so I end up with code that looks something like this:

(defn output-tap [path] (hfs-textline (str (get-prop :output-path-prefix) path) :sinkmode :replace))

(def some-cascalog-query 
    (<- [?f1 ?f2 ?f3] 
        ((output-tap (get-prop :output-path)) ?line)
        (tab-split ?line :> ?f1 ?f2 ?f3)))

In the example above, assume the function get-prop exists; it’s just using standard java to read a property value (based off this example: loading configuration file in clojure as data structure).

Now I have a main method that loads the property values, e.g. something like:

(defn -main [& args] (do (load-props (first args)) (do-cascalog-stuff)))

But when I lein uberjar I get a compile time error saying:

Caused by: java.lang.IllegalArgumentException: Can not create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
at org.apache.hadoop.fs.Path.<init>(Path.java:90)
at cascading.tap.hadoop.Hfs.getPath(Hfs.java:343)

Are defs always compile time evaluated (rather than runtime evaluated)? Or am I misunderstanding this error?

Answer 1

So, you want the property lookup to occur at run-time? Then yes, you’ll need to define some-cascalog-query as a function or macro. A bare def causes the expression to be evaluated when the code is loaded, not when the var is dereferenced.

This can be illustrated pretty simply in the REPL:

user=> (def foo (do (println "Hello, world!") 1))
Hello, world!
#'user/foo
user=> foo
1

From the documentation (emphasis mine):

(def symbol init?)

Creates and interns or locates a global var with the name of symbol and a namespace of the value of the current namespace (ns). If init is supplied, it is evaluated, and the root binding of the var is set to the resulting value.