Question

I realize this is a somewhat general question, but bear with me. Suppose that I have to insert a bunch of items into a data structure.

• I can either insert them in one at a time, or in one giant batch.
• The insert operation must take a lock on my data structure.
• The insertion happens from one thread.
• I don’t have any information about contention for the lock from other parts of the system.

Do you think it would be better to insert in batches or one-by-one or it doesn’t matter (…where “better” minimizes insertion time and lock hold time)?

Answer 1

If only one thread has access to your data structure, then you don’t need the lock at all.

If multiple threads need access then it’s a matter of how large is the “dosage”:

• If you lock for each item, then you have very fine grain locking and multiple threads can probably update the list alternatively, allowing all of them to make progress. However, locking and unlocking many times will start to add up.
• If you lock for the entire batch, then locking has negligible impact, but if the batch is large the other threads will be locked-out until you finish the entire update.
• A trade-off would probably be to divide the large batch into a few smaller batches, such that you don’t take the lock too many times, yet all the threads are allowed to make some progress by having a chance to take the lock between batch updates.