Question

I have a FileSystemWatcher set to check for new files, store the content in a database and delete the files. About 10 days ago, it started to ignore some files. We’re talking about 1,500 files from a total of 50,000 files. By manually moving the files to a different directory and then moving them again to the watched directory, the files get noticed.

The InternalBufferSize is set to 32 kB to deal with large batches. It processes 300+ files at once without a problem, and reality isn’t even close to that.

The program was last touched over 40 days ago, for a change unrelated to the FileSystemWatcher. It’s been in production for over a year now. No spikes can be seen in the server load.

What can cause an issue like this one to suddenly appear? Is there a possibility that FileSystemWatcher is simply unreliable?

Edit
I’ve created a test where 1,000 files are created. After running it, 3,000 entries can be found in the event log. So I believe buffer overflow is out of the question?

    private void button1_Click(object sender, EventArgs e)
    {
        fsw = new FileSystemWatcher();
        fsw.Path = @"C:\temp\fsw-test";
        fsw.IncludeSubdirectories = false;
        fsw.NotifyFilter = NotifyFilters.FileName;
        fsw.Created += new FileSystemEventHandler(fsw_Created_handler);
        fsw.EnableRaisingEvents = true;
        fsw.InternalBufferSize = 32768;
        fsw.Error += fsw_Error_handler;
    }

    private void fsw_Created_handler(object sender, FileSystemEventArgs e)
    {
        new Thread(new ParameterizedThreadStart(work)).Start(e);
    }

    private void fsw_Error_handler(object sender, ErrorEventArgs e)
    {
        EventLog.WriteEntry("few test", e.GetException().Message);
    }

    private void work(object e)
    {
        try
        {
            EventLog.WriteEntry("fsw test", "Queueing File Started");
            Thread.Sleep(10000);
            EventLog.WriteEntry("fsw test", ((FileSystemEventArgs)e).Name);
            EventLog.WriteEntry("fsw test", "Queueing File Done");
        }
        catch (Exception ex)
        {
            EventLog.WriteEntry("fsw test", "Error = " + ex.StackTrace + " *** " + ex.ToString());
        }
    }

    private void button2_Click(object sender, EventArgs e)
    {
        for (int i = 1; i <= 1000; i++)
        {
            System.IO.File.Create(@"C:\temp\fsw-test\" + i);
        }
    }

Edit 2
Stress testing the program in multiple ways and checking the code over and over again revealed no issues. So right now it’s an unreproducible bug, I’ll do a few changes to make it log more frequently and monitor the situation.

Answer 1

The problem turned out to be a Queue corruption, caused by multi-threading.