I’m writing a web service which, for each method call, must write a log entry into database. At a specific time, the call to this method may be very intensive (up to 1-2k request/minute), and our server is not so strong.
I want to use a List<Log> to cache the log entries, and then:
- batch insert 30-40 rows to the database, which greatly reduces the overhead
- when no more requests in more than 30 secs or 1 minute, all of the remaining cache will be written to the database.
The first condition is OK, but I don’t know how to implement for the second condition.
How can I do this, without adding too much overhead to my web service?
EDIT: I solved this based on Wheat’s suggestion.
- For each log entry, I send it directly to MSMQ queue and forget about it
- An separated service run continuously, take all of log entries are currently in the queue, bulk insert them to database, and then sleep for 30 seconds.
MSMQ was very helpful in this case!
You could use MSMQ or SQL Server Service Broker.