Question

I am using VSTS 2008 + C# + .Net 3.5 to develop a console application. I need to enumerate the most recent 50 files in current folder (to read file content, to get file meta data like file name, creation time, etc.). Since the current folder has about 5,000 files, and if I use Directory.GetFiles API, all 5,000 files’ meta data information will be read into memory. I think it is a waste since I only need to access the most recent 50 files.

Any solutions to access only the 50 most recent files in current directory?

Answer 1

This solution still loads metadata about all files, but I would say it’s fast enough for most uses. The following code reports that it takes around 50ms to enumerate the 50 most recently updated files in my Windows\System32 directory (~2500 files). Unless the code is run very frequently I would probably not spend time optimizing it a lot more:

FileInfo[] files = (new DirectoryInfo(@"C:\WINDOWS\System32")).GetFiles();
Stopwatch sw = new Stopwatch();
sw.Start();
IEnumerable<FileInfo> recentFiles = files.OrderByDescending(
                                              fi => fi.LastWriteTime).Take(50);
List<FileInfo> list = recentFiles.ToList();
sw.Stop();
Console.WriteLine(sw.ElapsedMilliseconds);
list.ForEach(fi => Console.WriteLine(fi.Name));

Update

Based on the discussion in the comments regarding using date/time in the file name: note that Directory.GetFiles does not load metadata about files; it simply returns a string array with file names (DirectoryInfo.GetFiles on the other hand returns an array of FileInfo objects). So, if you have date and time in your file names (preferably in a format that lends itself to sorting, such as yyyyMMdd-HHmmss or something like that) you can use Directory.GetFiles to get the file names, sort descending and then pick the 50 first from the list:

string[] files = Directory.GetFiles(pathToLogFiles);
IEnumerable<string> recentFiles = files.OrderByDescending(s => s).Take(50);
List<string> recentFiles = recentFiles.ToList();

Then loop over the list and load whatever data you need from each file.