Question

I have a data dump in a JSON file that I’m trying to pull into my app, but I think it needs some pre-processing, just not sure the best way to do this.

The format of the file is such that the data isn’t one JSON array, rather each line is a complete JSON array. Looks like:

{"username": "Bob", "id": 123456}
{"username": "Jane", "id": 456123}
{"username": "Paul", "id": 789456}
{"username": "Mary", "id": 159794}

No comma delimitations at the end of a line – no brackets, etc. When I try to pull this in, say through my seed JSON.parse it throws and error – doesn’t like the lack of appropriate formatting and wants it as one complete JSON dataset rather then each line being its own set.

What’s the best way to pull in this data? Thanks!

Answer 1

I’m not sure what you mean to do with the data after importing it, I assume you want to insert it into a database.

I would write a rake task like this:

task :import_users do
  File.open('/path/to/your.file', 'r') do |file|
    file.each do |line|
      user_attrs = JSON.parse line
      User.create! user_attrs
    end
  end
end