Question

I’m developing a small C# application that scans a log file for lines containing certain keywords and alerts the user when one of the keywords is found. This log is potentially extremely large (several gigabytes, in worst case scenario) but the only lines on the log that are relevant to me, are the ones added to the log while my application is running.

Is there a way I can capture each text line being appended to the file, without having to worry about the file content that was already present?

I already found out about the FileSystemWatcher class while searching for a solution, and while that seems great for notifying when I have new content to fetch from the log, it doesn’t seem to help for telling me what was added to it.

Answer 1

If you keep a FileStream open in Read mode (allowing writers, of course), you should be able to initially scan through the whole file and wait at the end until the FSW notifies you that the file has been modified.

Just be careful to reset your reading thread somehow if the file is deleted, for example if the log file that you are tailing gets rolled.

Here, I knocked together an example- run this, and while it is running, edit C:\Temp\Temp.txt in notepad and save it:

    public static void Main()
    {
        var lockMe = new object();
        using (var latch = new ManualResetEvent(true))
        using (var fs = new FileStream(@"C:\Temp\Temp.txt", FileMode.OpenOrCreate, FileAccess.Read, FileShare.ReadWrite))
        using (var fsw = new FileSystemWatcher(@"C:\Temp\"))
        {
            fsw.Changed += (s, e) =>
                               {
                                   lock (lockMe)
                                   {
                                       if (e.FullPath != @"C:\Temp\Temp.txt") return;
                                       latch.Set();
                                   }
                               };
            using (var sr = new StreamReader(fs))
                while (true)
                {
                    latch.WaitOne();
                    lock (lockMe)
                    {
                        String line;
                        while ((line = sr.ReadLine()) != null)
                            Console.Out.WriteLine(line);
                        latch.Set();
                    }
                }
        }
    }