Question

Update: I be glad to drop the C# requirement, and just see any program that can list all the files running as Admin or System, my question is has anyone seen such a thing?

There are numerous methods of enumerating files in a directory, but all suffer the same problems:

“The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.”

“Access to the path ‘C:\Users\All Users\Application Data’ is denied”

etc.

Even running under admin, single user machine, it seems impossible to list all the files without encountering exceptions\errors.

Is it really an impossible task just to get list of all the files under windows? Has anyone ever been able to obtain the complete list of all files on their machine using C# or any other method?

This link from MS with the title “Enumerate Directories and Files” , does not show how to Enumerate Directories and Files, it only show a subset of what that will not throw : DirectoryNotFoundException, UnauthorizedAccessException, PathTooLongException,

Update : Here is sample code to run over C and attempt to enumerate all the files and errors. Even when running this as admin there are folders that not only can be access, but I even can’t change their ownership to Admin! for example : “C:\Windows\CSC”

just have look at “Errors {0}.csv” log file to see how many places are inaccessible to admin.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;


class Program
{

static System.IO.StreamWriter logfile;
static System.IO.StreamWriter errorfile;
static void Main(string[] args)
{
    string directory = @"C:\";

    logfile = new System.IO.StreamWriter(string.Format(@"E:\Files {0}.csv", DateTime.Now.ToString("yyyyMMddHHmm")));
    errorfile = new System.IO.StreamWriter(string.Format(@"E:\Errors {0}.csv", DateTime.Now.ToString("yyyyMMddHHmm")));
    TraverseTree(directory, OnGotFileInfo, OnGotException);

    logfile.Close();
    errorfile.Close(); 
}

public static void OnGotFileInfo(System.IO.FileInfo fileInfo)
{
    logfile.WriteLine("{0},{1},", fileInfo.FullName, fileInfo.Length.ToString("N0"));
}

public static void OnGotException(Exception ex, string info)
{
    errorfile.WriteLine("{0},{1}", ex.Message, info);
}

public static void TraverseTree(string root, Action<System.IO.FileInfo> fileAction, Action<Exception, string> errorAction)
{
    // Data structure to hold names of subfolders to be 
    // examined for files.
    Stack<string> dirs = new Stack<string>(20);

    if (!System.IO.Directory.Exists(root))
    {
        throw new ArgumentException();
    }
    dirs.Push(root);

    while (dirs.Count > 0)
    {
        string currentDir = dirs.Pop();
        string[] subDirs;
        try
        {
            subDirs = System.IO.Directory.GetDirectories(currentDir);
        }
        // An UnauthorizedAccessException exception will be thrown if we do not have 
        // discovery permission on a folder or file. It may or may not be acceptable  
        // to ignore the exception and continue enumerating the remaining files and  
        // folders. It is also possible (but unlikely) that a DirectoryNotFound exception  
        // will be raised. This will happen if currentDir has been deleted by 
        // another application or thread after our call to Directory.Exists. The  
        // choice of which exceptions to catch depends entirely on the specific task  
        // you are intending to perform and also on how much you know with certainty  
        // about the systems on which this code will run. 

        catch (System.Exception e)
        {
            errorAction(e, currentDir);
            continue;
        }

        string[] files = null;
        try
        {
            files = System.IO.Directory.GetFiles(currentDir);
        }

        catch (System.Exception e)
        {
            errorAction(e, currentDir);
            continue;
        }

        // Perform the required action on each file here. 
        // Modify this block to perform your required task. 
        foreach (string file in files)
        {
            try
            {
                // Perform whatever action is required in your scenario.
                System.IO.FileInfo fi = new System.IO.FileInfo(file);
                fileAction(fi);
            }
            catch (System.Exception e)
            {
                // If file was deleted by a separate application 
                //  or thread since the call to TraverseTree() 
                // then just continue.
                errorAction(e ,file);
                continue;
            }
        }

        // Push the subdirectories onto the stack for traversal. 
        // This could also be done before handing the files. 
        foreach (string str in subDirs)
            dirs.Push(str);
    }

    }
}

Answer 1

Yes, it is at very least hard to enumerate all files without exceptions.

Several set of issues here:

• some path (long ones- PathTooLongException) are not supported by CLR
• security restrictions on folders/files
• junctions/hard links that introduce duplicates (and in theory cycles to case StackOverflow in recursive iteration).
• basic sharing violation restrictions (if you try to read files).

For PathTooLongException: I think you’ll need to deal with PInvoke of corresponding Win32 functions. All path related methods in CLR are restricted to 256 characters long.

Security restrictions – you may be able to enumerate everything if you run under system (not sure) or with backup permissions, but any other account is guaranteed to not being able to access all files on system configured by default.
Instead of getting exceptions you can PInvoke native versions and handle error codes instead. You may be able to decrease number of exceptions on going into directories by checking ACL on the directly first.