Question

I recently got put on a project where they’re having issues with too many files in a folder slowing down access. I believe it is 10,000+ files in a single folder where windows starts to slow down access, we have something on the order of 50,000. All the files are small and most of the time we only need to access the newest .1-2.% of them via windows file and print sharing. I’d look into dividing the files into subfolders, except that there is a bunch of legacy code that is only able to look at a single folder.

My idea – I don’t know if it is possible or even plausible – is to create a small program that buffers the newest .1-.2% files in memory, and retrieves the rest from disk as needed.

1. I had thought that years ago I’d read of a protocol that could simulate a folder on a hard drive. Is it possible?

2. Is there something out there that already does this? Is there a better option without major changes to the system?

3. What to other systems use for serving up a large number of files? Is there some other product that serves files that we could map as a network drive? Or some way to blend 2 folders so they look like one?

Answer 1

Putting aside the “correct way to solve this problem” for the moment, what you’re looking for is called “Shell namespace extensions“. There are several .NET resources for writing these explorer extensions.

• http://namespaceextension.codeplex.com/
• http://www.codeproject.com/Articles/1649/The-Complete-Idiot-s-Guide-to-Writing-Namespace-Ex
• http://www.codeproject.com/Articles/13515/A-Namespace-Extension-Toolkit
  And perhaps many more.

Of course – we must remember why it isn’t a good idea to write explorer extensions in .NET.

Hope this helps.