Question

I’m doing some data analytics aggregations and here is my data structures:

{
12300 {
   views {
       page-1 {
          link-2  40
          link-6  9
       }
       page-7 {
          link-3  9
          link-11 8
       }
   }
   buttons {
       page-1 {
          link-22 2
       }
   }
}
34000      ....
}

Where 12300, 34000 are a time values.

What I want to do is to traverse that data structure and insert entries into a database, something like this:

insert into views (page, link, hits, time) values (page-1, link-2, 40, 12300)
insert into views (page, link, hits, time) values (page-1, link-6, 9, 12300)

What would be an idiomatic way to code that? Am I complicating the data structure? do you suggest any better way to collect the data?

Answer 1

Assuming you have a jdbc connection from clojure.java.jdbc, this should come close to what you want.

(jdbc/do-prepared "INSERT INTO views (page, link, hits, time) VALUES (?, ?, ?, ?)"
                  (for [[time data] m
                        [data-type page-data] data
                        [page links] page-data
                        [link hits] links]
                    [page link hits time]))
;; why aren't we using data-type, eg buttons?

Edit for clarified problem

(let [m '{12300 {views {page-1 {link-2  40
                                link-6  9}
                        page-7 {link-3  9
                                link-11 8}}
                 buttons {page-1 {link-22 2}}}
          34000 {views {page-2 {link-2 5}}}}]
  (doseq [[table rows] (group-by :table (for [[time table] m
                                              [table-name page-data] table
                                              [page links] page-data
                                              [link hits] links]
                                          {:table table-name, :row [page link hits time]}))]
    (jdbc/do-prepared (format "INSERT INTO %s (page, link, hits, time) VALUES (?, ?, ?, ?)" table)
                      (map :row rows))))