This problem is driving me a bit insane – I have spent the best of 3 days on it now. To make it a bit more annoying I cannot replicate it on our development/testing environments – it only seems to manifest within our production environment.
The scenario:
There is a page that fires off an uploader – this uploader is a YUI2 javascript uploader component that uses flash under the hood. The file uploads fine to our servers, after which an uploadsuccess javascript event fires which results in an ajax request to the server so that we can do further processing. The processing takes quite a while – up to 12 minutes. I know this is a problem in itself, but the work required to create an asynchronous threaded process for this is fairly complex and we have scheduled this for another day. For now we have increased the web request timeout limit within the Web.config so that the request doesn’t time out.
On our dev/test environments the process completes and a response is recieved.
On our production environment the process completes, but no response is ever recieved. 🙁
My attempt to find the problem:
I added an insane amount of logging to the code, and even dropped in logging points that logged when each event of the asp.net page lifecyle is hit.
The action seems to process correctly, then each of the page lifecycle events are hitting right up until the last one (the PreSendRequestContent which fires just before the content response is sent back to the client). However, I receive absolutely nothing back – I even had fiddler up and running to make sure it wasn’t some strange browser issue. Nothing. 🙁
I am a developer so I feel like I have exhausted my options in terms of debugging here. The problem seems to lie somewhere deeper in IIS, after it leaves the asp.net stack and before it returns back to the client.
My plea:
Does anyone out there have any remote sort of inclination of what could possibly be happening, or what I could do in order to continue my troubleshooting and debugging?
As stated in the tags, the application is an ASP.NET application (specifically MVC 2), running on an IIS 7.5 webserver on a Windows Server 2008 machine.
I will praise you unto the end of days if you can help in any way.
Your sincerely;
The frustrated developer
Ok, so the problem has been solved, however I was never able to isolate the exact issue and point in which the request was being dropped. It was an intermittent issue that was only being hit during VERY long running requests, which is always a bad idea for the web anyways.
I solved this issue by creating a background worker queue with a number of worker threads. Requests to the original action are now farmed off to the worker queue. I have also exposed a status page where a user can track the progress of these items, and even retry the failed items.
Seems to work well, and the UI is way more responsive, which is always a good thing.