Question

Consider the follow code that downloads a file from internet using Indy components:

procedure TForm26.Button1Click(Sender: TObject);
var
  vFileStream : TStream;
begin
  DeleteFile('C:\test.exe');
  vFileStream := TFileStream.Create('C:\test.exe', fmCreate);
  IdHTTP1.Get('SomeUrl', vFileStream);
  vFileStream.Free;
end;

I am getting an Out Of memory exception. What is happening is that besides the fact that I am using TFileStream, the bytes that are being written to it do not go directly to the disk, instead, they stay in memory until get terminates.

The file I am trying to download is really, really big.

Does anyone knows how to download a big file without getting an out of memory exception?

Delphi 2010 and latest Indy 10 from Indy’s SVN.

EDIT

That is not a FileStream problem. It is a Indy problem. Indy is, somehow, caching the file in memory before it writes to the stream.

Answer 1

TIdHTTP downloads the entire file into memory if the data is compressed, or if the data is HTML and the TIdHTTP.HTTPOptions property does not contain the hoNoParseMetaHTTPEquiv flag.

Indy does not yet support streaming decompression for HTTP (it does for FTP, though), so TIdHTTP caches the entire compressed data in memory before it can then decompress it to the file.

Parsing HTML is sometimes needed in cases where the HTML overrides HTTP header values with new values via HTML <meta> tags, most importantly the data’s Charset value, so TIdHTTP can decode the data using the correct charset when the data is being returned to user’s code as a String. Enabling the hoNoParseMetaHTTPEquiv flag disables that parsing, and thus any caching of HTML data (unless compression is also used, that is).