Question

I have an application that logs information to a daily text file every second on a master PC. A Slave PC on the network using the same application would like to copy this text file to its local drive. I can see there is going to be file access issues.

These files should be no larger than 30-40MB each. the network will be 100MB ethernet. I can see there is potential for the copying process to take longer than 1 second meaning the logging PC will need to open the file for writing while it is being read.

What is the best method for the file writing(logging) and file copying procedures? I know there is the standard Windows CopyFile() procedure, however this has given me file access problems. There is also TFileStream using the fmShareDenyNone flag, but this also very occasionally gives me an access problem too (like 1 per week).

What is this the best way of accomplishing this task?

My current File Logging:

procedure FSWriteline(Filename,Header,s : String);
var LogFile : TFileStream;
line : String;
begin
     if not FileExists(filename) then
     begin
          LogFile := TFileStream.Create(FileName, fmCreate or fmShareDenyNone);
          try
             LogFile.Seek(0,soFromEnd);
             line := Header + #13#10;
             LogFile.Write(line[1],Length(line));
             line := s + #13#10;
             LogFile.Write(line[1],Length(line));
          finally
                 logfile.Free;
          end;
     end else begin
         line := s + #13#10;
         Logfile:=tfilestream.Create(Filename,fmOpenWrite or fmShareDenyNone);
         try
            logfile.Seek(0,soFromEnd);
            Logfile.Write(line[1], length(line));
         finally
            Logfile.free;
         end;
     end;
end;

My file copy procedure:

procedure DoCopy(infile, Outfile : String);
begin
     ForceDirectories(ExtractFilePath(outfile)); //ensure folder exists
     if FileAge(inFile) = FileAge(OutFile) then Exit; //they are the same modified time
     try
        { Open existing destination }
        fo := TFileStream.Create(Outfile, fmOpenReadWrite or fmShareDenyNone);
        fo.Position := 0;
     except
           { otherwise Create destination }
           fo := TFileStream.Create(OutFile, fmCreate or fmShareDenyNone);
     end;
     try
        { open source }
        fi := TFileStream.Create(InFile, fmOpenRead or fmShareDenyNone);
        try
           cnt:= 0;
           fi.Position := cnt;
           max := fi.Size;
           {start copying }
           Repeat
                 dod := BLOCKSIZE; // Block size
                 if cnt+dod>max then dod := max-cnt;
                 if dod>0 then did := fo.CopyFrom(fi, dod);
                 cnt:=cnt+did;
                 Percent := Round(Cnt/Max*100);
           until (dod=0)
        finally
               fi.free;
        end;
     finally
            fo.free;
     end;
end;

Answer 1

I would suggest not closing and reopening the shared file over and over to begin with. Since you write to it every second, that is just needless overhead.

On the Master side, create and close the file (the fmCreate flag cannot be used with other flags!), then re-open it in fmOpenWrite mode with fmShareDenyWrite sharing, leave it open, and write to it when needed.

On the Slave side, open the file in fmOpenRead mode with fmShareDenyNone sharing, leave it open, and read from it every second. No need to copy the entire shared file over the network every time. That is wasted bandwidth. Just read whatever new data has been written in the past few seconds and that is all. If the Slave needs the data to be stored in a local file, then it can manage a separate local file independantly of the shared file, pushing new data into the local file when needed.