Question

The scenario:

user uploads a small file or about 100 kB to the ASP.NET 4.0 application with perhaps 1000 units of work.
The server then gets to work on the units of work, one at a time.
Each unit takes a few seconds to complete due to requesting information from an external service.
The results are only saved to the database if all units were completed successfully, using a transaction.
Once completed, the user may get a list of what was done.

The problem is that the user does not get the confirmation, instead his browser I think gives up because of a timeout.

Future files are expected to be a few 100 times larger, increasing the problem.

I want to prevent this timeout.

Here are some ideas I had:

1. Optimize the code to run faster. This is done and is no longer the problem.
2. Run the requests to the external service in parallel.
3. Increase the server timeouts a little.
4. Let the user upload the file and then send him an email with the results later when the file has been processed.
5. Somehow make the page refresh and show some progress information to the user while waiting, e.g., 5% complete – done in 10 minutes.

How could I implement this last step, showing progress information and preventing browser timeout?

Other suggestions are welcome.

Thanks,

Answer 1

You put the data uploaded into a queue in your database. Then, you have an asynchronous process (example Windows Service) to pull items from your queue and process them. You can update your DB with progress of each operation, and when full completed, remove the item from the queue and update your other tables.

For progress, the user can then query the queue table for the status of his upload.