Question

I have this delphi code that basically download a file (using Delphi 2010 + Indy 10.5.8 r4743), everything works just fine, except that when I download 100Mb (for example), it seems that Indy assign the full size (ie. a file with 100MB of dummy content(*) is instantly created), then download the file.

In the end the 100MB is correctly downloaded, but since the download process is done in the background using a hidden EXEecutable, I based my code to rely on the temporary file size to update the main UI

with IdHTTP do
begin
     if FileExists(LocalFile) then
        iLength     := FileSize2(LocalFile)
     else
         iLength    := 0;

     DoExit         := False;
     try
        try
           repeat
                 if ExitApp then
                    Exit;

                 if Not FileExists(LocalFile) then
                    AFileStream     := TFileStream.Create(LocalFile, fmCreate)
                 else
                 begin
                      // File already exist, resume download
                      AFileStream   := TFileStream.Create(LocalFile, fmOpenReadWrite);
                      DoExit        := (AFileStream.Size >= iLength);
                      if (Not DoExit) then
                         AFileStream.Seek(Max(0, AFileStream.Size - 4096), soFromBeginning);
                 end;

                 iRangeEnd          := AFileStream.Size + 50000;

                 if (iRangeEnd < iLength) then
                    Request.Range   := IntToStr(AFileStream.Position) + '-' + IntToStr(iRangeEnd)
                 else
                 begin
                      Request.Range := IntToStr(AFileStream.Position) + '-';
                      DoExit        := True;
                 end;

                 PostTime           := Now;
                 Get(TheURL, AFileStream);
                 IsError            := Not (ResponseCode in [200, 206]);
           until DoExit;

           Disconnect;
        except
              IsError               := True;
        end;    // try/except
     finally
            FreeAndNil(AFileStream);
     end;    // try/finally
end;    // with

My question is Is there a way to avoid this behavior from indy? I know I can use the OnWork event, but then I will need to keep track of file names.

Ideally, I’d also like to avoid IPC (kinda overkill + I don’t want to use that, say every second, for multiple downloads, I very much prefer to use the file size as indication of download progress as it gives more freedom when updating the UI)

(*) I assume it’s dummy content since I need between 60-100 seconds on my current internet connection speed to really get the file

Answer 1

Yes, TIdHTTP pre-allocates the full file size if it knows the size ahead of time, based on the HTTP response headers. This is an optimization technique. Pre-allocating the file avoids unnecessary file system overhead when writing larger files, since the file does not have to keep growing over time, hunting for available sectors on the HDD, slowing down the writing process. There is currently no way to disable the pre-allocation, it is hard-coded behavior in Indy’s internals. So relying on the file’s actual size as a progress indicator will not work for you. You are going to have to communicate the actual progress information from your background app to the main app.