Question

I need to programmatically download a large file before processing it. What’s the best way to do that? As the file is large, I want to specific time to wait so that I can forcefully exit.

I know of WebClient.DownloadFile(). But there does not seem a way to specific an amount of time to wait so as to forcefully exit.

try
{
    WebClient client = new WebClient();
    Uri uri = new Uri(inputFileUrl);
    client.DownloadFile(uri, outputFile);
}
catch (Exception ex)
{
    throw;
}

Another way is to use a command line utility (wget) to download the file and fire the command using ProcessStartInfo and use Process’ WaitForExit(int ms) to forcefully exit.

ProcessStartInfo startInfo = new ProcessStartInfo();
//set startInfo object

try
{
    using (Process exeProcess = Process.Start(startInfo))
    {
        //wait for time specified
        exeProcess.WaitForExit(1000 * 60 * 60);//wait till 1m

        //check if process has exited
        if (!exeProcess.HasExited)
        {
            //kill process and throw ex
            exeProcess.Kill();
            throw new ApplicationException("Downloading timed out");
        }
    }
}
catch (Exception ex)
{
    throw;
}

Is there a better way? Please help. Thanks.

Answer 1

Use a WebRequest and get the response stream. Then read from the reponse Stream blocks of bytes, and write each block to the destination file. This way you can control when to stop if the download takes too long, as you get control between chunks and you can decide if the download has timed out based on a clock:       

        DateTime startTime = DateTime.UtcNow;
        WebRequest request = WebRequest.Create("http://www.example.com/largefile");
        WebResponse response = request.GetResponse();
        using (Stream responseStream = response.GetResponseStream()) {
            using (Stream fileStream = File.OpenWrite(@"c:\temp\largefile")) { 
                byte[] buffer = new byte[4096];
                int bytesRead = responseStream.Read(buffer, 0, 4096);
                while (bytesRead > 0) {       
                    fileStream.Write(buffer, 0, bytesRead);
                    DateTime nowTime = DateTime.UtcNow;
                    if ((nowTime - startTime).TotalMinutes > 5) {
                        throw new ApplicationException(
                            "Download timed out");
                    }
                    bytesRead = responseStream.Read(buffer, 0, 4096);
                }
            }
        }