Question

My application allows user to upload CSV file that is processed and records are written in the database. But this file can contain a very big number of records for example 300 000. And in this case it may need to up to half an hour to process all this records, I would like my application not to freeze the page for this period, but show progress and maybe some errors, or it would be better to allow user to move to another pages and from time to time come back to check process.
By what means can I achieve that?

Answer 1

The approach we took to resolve a similiar issue was as follows;

Upload File using normal http methods.

Save file locally.

Submit file to asynchronous webservice (.asmx). This process will insert a record that will store the status of the import, along with actually starting importing the records. Once all records have been processed, set the status accordingly.

This all happens in a single flow. Because the WebMethod is asynchronous, it will return without waiting for itself to complete and the import will happen in the background.

You now redirect user to page that periodically checks the status of the asynchronous import until such point as it is finished. You can also add additional information to this process such as progress by batching the records and updating another fields accordingly.

This has worked well for us for many years now. I have not added any real detail as that will be specific to your implementation.