Question

I am learning ruby on rails.
I started by importing some data into a sqlite database from a csv file. Then I successfully transferred that data into my rails environment.
Upon inspection of the database, I realized that I had created 5 copies of each entry in the database. I wanted to clean the database and I was wondering what the best options to do that would be?

Here is what I guess I need to do, but please suggest better ways if you think of them:

1. Write a method in rails that invokes raw sql that removes the possible duplicates from the table and enters them into another table called “duplicates”
2. Then go through the entries in table “Duplicates” and decide whether to keep them or delete them.
3. Finally after the check is done, transfer the entries to be retained back to the original table

Also, where should I put this method to remove duplicates? In the “model” or somewhere else?

Answer 1

The easiest solution is just clear your database and re-import so you have just one copy.

Or you could use an SQLite client and clean it up in SQL directly.

For a utility method like this, if you choose that route, generally you would make a Rake task. So it would go in lib/tasks.