Question

I create a separate collection nightly refreshing data, and the first thing I do is remove all the entries which are currently in it. Through profiling I find that this remove command can take upwards of 101190 milliseconds..

Is this normal? Is there something I should be doing in order to speed this query up? it seems that just emptying a collection should be fairly instant.

Thank you for your help!

Answer 1

Doing a remove() with no filter condition can be very slow, since each document must be removed from the collection data files and any indexes which exist (including the default index on _id).

If you are removing all the documents in a collection, and especially if you are only using the default index or you can re-create your indexes easily from within your application code, you can drop() the collection instead, and then begin adding your new data.

You can further minimize the time that the collection appears to be empty or partially full by using the renameCollection command like (in pseudo-Python):

for new_record in new_records:
    db.newcollection.insert(new_record)

for index_spec in index_specs:
    db.newcollection.ensure_index(index_spec)

db.oldcollection.drop()
db.command('renameCollection', 'newcollection', to='oldcollection')