Question

I’m grabbing some json from here on freebase (careful, you can only request this a few times without using &key=your-key).

I want to convert the response into something similar to this:

    ({:case "Roe v. Wade", :plaintiffs ("Norma McCorvey"), :defendants ("Henry Wade"), :court "Supreme Court of the United States", :subjects ("Abortion" "Privacy"), :article "http://wp/en/68493"} ...)

Here’s the code I came up with after using clojure.data.json/read-string:

    (defn extract-data [case]
      {:case (case "name")
       :plaintiffs (flatten (map #(get % "parties") (filter (fn [p] (some #(= (% "id") "/en/plaintiff") (p "role")))
                                                           (case "/law/legal_case/parties"))))
       :defendants (flatten (map #(get % "parties") (filter (fn [p] (some #(= (% "id") "/en/defendant") (p "role")))
                                                           (case "/law/legal_case/parties"))))
       :court (get-in case ["court" 0 "name"])
       :subjects (map #(% "name") (case "subject"))
       :article (get-in case ["/common/topic/article" 0 "source_uri" 0])})

    (def response (-> query-uri
                       java.net.URL.
                       slurp
                       json/read-str))
    (def case-data (map extract-data (response "result")))

extract-data seems overly complex though, is there a better way to do this? Is this a case where core.logic could be used? If so, how?

Answer 1

You may have a look a different query systems (zip-filters, core.logic, datomic’s datalog on collections etc.). Or roll your own ad-hoc one:

(defn select [x path]
  (if-let [[p & ps] (seq path)]
    (if (fn? p)
      (mapcat #(select % ps) (filter p x))
      (recur (get x p) ps))
    x))

(def mapping
  {:case ["name"]
   :plaintiffs ["/law/legal_case/parties"
                #(= (get-in % ["role" 0 "id" 0]) "/en/plaintiff")
                "parties"]
   :defendants ["/law/legal_case/parties"
                #(= (get-in % ["role" 0 "id" 0]) "/en/defendant")
                "parties"]
   :court ["court" 0 "name" 0]
   :subjects ["subject" (constantly true) "name"]
   :article ["/common/topic/article" 0 "source_uri" 0]})

(defn extract-data [x mapping]
  (into {}
    (for [[k path] mapping]
      [k (if (some fn? path) (select x path) (get-in x path))])))

And then (map #(extract-data % mapping) results) should do the trick

=> (extract-data (first result) mapping)
{:case "Roe v. Wade", :plaintiffs ("Norma McCorvey"), :defendants ("Henry Wade"), :court "Supreme Court of the United States", :subjects ("Abortion" "Privacy"), :article "http//wp/en/68493"}

This type of code (the query interpreter) may be brittle, so be sure to have a test suite.