Question

Lets say. If i have 1,000,000 users registered on my system and I have a list of 500 names.

I would like to crosscheck these names against the number of users on my system to see which names / users are already registered on the db.

Will such a process slow down the app significantly? Or does this sort of thing happen all the time?

Perhaps I can cache the results every 30 mins so I don’t have to call it every time.

EDIT >> A bit of clarification:

I forgot to mention that I am trying to udate the list of 500 names. So if ‘foobar’ and ‘joe’ is on this list and is also registered in the db, then all I want to do is remove ‘foobar’ and ‘joe’ from the list, giving me 498 names.

I don’t think it would be suitable to do something like:

User.where('name in (?)', Array('foobar', 'joe'))

I would have do something like:

User.each do |registered_user|
  index = list.index(list.find{ |user| user.screen_name.downcase == registered_user.screen_name.downcase })
  list.delete_at(index) if index
end

filtered_list = list

Is above code, overkill?

Answer 1

If you do it cleverly, it can be very fast. Let the database do the work.

SELECT * FROM USERS WHERE Name IN ( <generated list of your 500 user name candidates>)

The returned list will be the ones of the 500 which already exist in the database. Such as call will be fast, at most a few seconds on a slow database.

Cheers,
Daniel