Question

Let’s say that you want to write an application that processes multiple text files, supplied as arguments at the command line (e.g., MyProcessor file1 file2 …). This is a very common task for which Perl is often used, but what if one wanted to take advantage of .NET directly and use C#.

What is the simplest C# 4.0 application boiler plate code that allows you to do this? It should include basically line by line processing of each line from each file and doing something with that line, by either calling a function to process it or maybe there’s a better way to do this sort of “group” line processing (e.g., LINQ or some other method).

Answer 1

You could process files in parallel by reading each line and passing it to a processing function:

class Program
{
    static void Main(string[] args)
    {
        Parallel.ForEach(args, file =>
        {
            using (var stream = File.OpenRead(file))
            using (var reader = new StreamReader(stream))
            {
                string line;
                while ((line = reader.ReadLine()) != null) 
                {
                    ProcessLine(line);
                }
            }
        });
    }

    static void ProcessLine(string line)
    {
        // TODO: process the line
    }
}

Now simply call : SomeApp.exe file1 file2 file3

Pros of this approach:

• Files are processed in parallel => taking advantage of multiple CPU cores
• Files are read line by line and only the current line is kept into memory which reduces memory consumption and allows you to work with big files