Question

I’m writing an application that uploads large files to a web service using HttpWebRequest.

This application will be run by various people with various internet speeds.

I asynchronously read the file in chunks, and asynchronously write those chunks to the request stream. I do this in a loop using callbacks. And I keep doing this until the whole file has been sent.

The speed of the upload is calculated between writes and the GUI is subsequently updated to show said speed.

The issue I’m facing is deciding on a buffer size. If I make it too large, users with slow connections will not see frequent updates to the speed. If I make it too small, users with fast connections will end up “hammering” the read/write methods causing CPU usage to spike.

What I’m doing now is starting the buffer off at 128kb, and then every 10 writes I check the average write speed of those 10 writes, and if it’s under a second I increase the buffer size by 128kb. I also shrink the buffer in a similar fashion if the write speed drops below 5 seconds.

This works quite well, but it all feels very arbitrary and it seems like there is room for improvement. My question is, has anybody dealt with a similar situation and what course of action did you take?

Thanks

Answer 1

I think this is a good approach. I too used in large file upload. But there was a small tweek in that. I determined the connection speed in the first request by placing a call to my different service. This would actually save the overhead of recalculating the speed with every request. The primary reason of doing so was

1. In slow connection, the speed usually fluctuate very much. Thus recalculating it every request does not make sense.

2. I was supposed to provide resume facility also where the user would be able to reupload the file from the point where it ended last time.

Considering the scalability, I used to fix the buffer with the first request. Let me know if it helped