Question

I’m writing a little parser in clojure for learning purpose.
basically is a TSV file parser that need to be put in a database, but I added a complication.
The complication itself is that in the same file there are more intervals.
The file look like this:

###andreadipersio 2010-03-19 16:10:00###                                                                                
USER     COMM               PID  PPID  %CPU %MEM      TIME  
root     launchd              1     0   0.0  0.0   2:46.97  
root     DirectoryService    11     1   0.0  0.2   0:34.59  
root     notifyd             12     1   0.0  0.0   0:20.83  
root     diskarbitrationd    13     1   0.0  0.0   0:02.84`
....

###andreadipersio 2010-03-19 16:20:00###                                                                                
USER     COMM               PID  PPID  %CPU %MEM      TIME  
root     launchd              1     0   0.0  0.0   2:46.97  
root     DirectoryService    11     1   0.0  0.2   0:34.59  
root     notifyd             12     1   0.0  0.0   0:20.83  
root     diskarbitrationd    13     1   0.0  0.0   0:02.84

I ended up with this code:

(defn is-header? 
  "Return true  if a line is header"
  [line]
  (> (count (re-find #"^\#{3}" line)) 0))

(defn extract-fields
  "Return regex matches"
  [line pattern]
  (rest (re-find pattern line)))

(defn process-lines
  [lines]
  (map process-line lines))

(defn process-line
  [line]
  (if (is-header? line)
    (extract-fields line header-pattern))
  (extract-fields line data-pattern))

My idea is that in ‘process-line’ interval need to be merged with data so I have something like this:

('andreadipersio', '2010-03-19', '16:10:00', 'root', 'launchd', 1, 0, 0.0, 0.0, '2:46.97')

for every row till the next interval, but I can’t figure how to make this happen.

I tried with something like this:

(def process-line
  [line]
  (if is-header? line)
    (def header-data (extract-fields line header-pattern)))
  (cons header-data (extract-fields line data-pattern)))

But this doesn’t work as excepted.

Any hints?

Thanks!

Answer 1

You’re doing (> (count (re-find #"^\#{3}" line)) 0), but you can just do (re-find #"^\#{3}" line) and use the result as a boolean. re-find returns nil if the match fails.

If you’re iterating over the items in a collection, and you want to skip some items or combine two or more items in the original into one item in the result, then 99% of the time you want reduce. This usually ends up being very straightforward.

;; These two libs are called "io" and "string" in bleeding-edge clojure-contrib
;; and some of the function names are different.
(require '(clojure.contrib [str-utils :as s]
                           [duck-streams :as io])) ; SO's syntax-highlighter still sucks

(defn clean [line]
  (s/re-gsub #"^###|###\s*$" "" line))

(defn interval? [line]
  (re-find #"^#{3}" line))

(defn skip? [line]
  (or (empty? line)
      (re-find #"^USER" line)))

(defn parse-line [line]
  (s/re-split #"\s+" (clean line)))

(defn parse [file]
  (first
   (reduce
    (fn [[data interval] line]
      (cond
       (interval? line) [data (parse-line line)]
       (skip? line)     [data interval]
       :else            [(conj data (concat interval (parse-line line))) interval]))
    [[] nil]
    (io/read-lines file))))