Question

How could I speed up this linq query?

It takes a long time and when I place a lot of objects in the list I get a memory exception.

List<DirectoryInfo> directoriesThatWillBeCreated = new List<DirectoryInfo>();
// some code to fill the list
// ..
// ..

List<FileInfo> FilesThatWillBeCopied = new List<FileInfo>();
// some code to fill the list
//....

directoriesThatWillBeCreated = (from a in FilesThatWillBeCopied
                                from b in directoriesThatWillBeCreated
                                where a.FullName.Contains(b.FullName)
                                select b).ToList();

I hope I can do something like previous solution but I don’t know how to do that when dealing with different types of objects. Do I have to create a new class then convert all the FileInfo and DirectoryInfo objects to that class then perform the query? Moreover FileInfo and DirectoryInfo classes are sealed and I cannot inherit from them therefore I’ll have to create a new class and that will be not to efficient. At least that will be more efficient than that query because that query takes forever.

Answer 1

One thing you could do is change the Contains to a StartsWith. StartsWith will fail faster in the event of a failed match.

directoriesThatWillBeCreated = (from a in FilesThatWillBeCopied
                                from b in directoriesThatWillBeCreated
                                where a.FullName.StartsWith(b.FullName)
                                select b).ToList();

This isn’t a complete solution, though. If FilesThatWillBeCopied has M items and directoriesThatWillBeCreated has N elements, then your query is going to process MxN string comparisons.

Another Option

Another optimization to try, iterate through directoriesThatWillBeCreated first, then select those that match any FileInfo in FilesThatWillBeCopied. By checking if any match, you could break out of testing the files once a match is found. That could be done like this: (warning, notepad code follows)

directoriesThatWillBeCreated = directoryThatWillBeCreated
    .Select(b => FilesThatWillBeCopied
    .Any(a => a.FullName.StartsWith(b.FullName)));