Question

This is a similar question I asked earlier, except different in the fact that I am taking multiple files and calculating the total sums off of those files. I have it to the point where I am reading all the files from a specific directory, but for some reason its not grouping correctly.

Here is the code that I have:

public void CalculateMonthlyStatistics(string monthlyFiles)
        {
            string monthlyFileName = monthlyFiles + ".log";

            var statistics = File.ReadLines(monthlyFileName)

            .GroupBy(items => items[0])
            .Select(g =>
            new
            {

                Division = g.Key,
                ZipFiles = g.Sum(i => Convert.ToInt32(i[1])),
                Conversions = g.Sum(i => Convert.ToInt32(i[2])),
                ReturnedFiles = g.Sum(i => Convert.ToInt32(i[3])),
                TotalEmails = g.Sum(i => Convert.ToInt32(i[4]))
            });

            statistics
               .ToList()
               .ForEach(d => Console.WriteLine("{0}\t{1}\t{2}\t{3}\t{4}", 
                        d.Division, 
                        d.ZipFiles, 
                        d.Conversions, 
                        d.ReturnedFiles,  
                        d.TotalEmails));
               Console.Read();
               //.ForEach(d => Log.Open(tempFileName.TrimEnd(charsToTrim), d.Division, d.ZipFiles, d.Conversions, d.ReturnedFiles, d.TotalEmails));
        }
    }
}

The log files that I am putting into it look like the following:

 Division   Zip Files   Conversions Returned Files  Total E-Mails   
Corporate   0   5   0   5   
Energy  0   1   0   5   
Global Operations   0   3   0   3   
Oil & Gas   1   5   0   5   
Capital 5   18  0   12  

So what I am trying to do, is group by “Corporate”, “Energy”, etc. Then calculate the totals for ALL of the files being read, to create a Monthly Statistics file. I am getting totals currently, however I think its got something to do with the header that I am passing in, and I am not sure how to tell it to skip that line.

Thanks in advance

EDIT

Here is my processor, which originally reads the directory, etc.

public void ProcessMonthlyLogFiles()
    {
        DateTime currentTime = DateTime.Now;

        int month = currentTime.Month - 1;
        int year = currentTime.Year;

        string path = Path.GetDirectoryName(Settings.DailyPath + year + @"\" + month + @"\");

        foreach (string monthlyFileNames in Directory.GetFiles(path))
        {
            string monthlyFiles = path + @"\" + Path.GetFileNameWithoutExtension(monthlyFileNames);
            new MonthlyReader().CalculateMonthlyStatistics(monthlyFiles);
        }
    }

The processor finds the proper directory to search through in order to get the files from. It uses the current date, and finds last month.

Answer 1

Skipping the header is straightforward:

File.ReadLines(monthlyFileName).Skip(1).<rest of your chain>

However, it seems as though you’re reading one file at a time, when you want to be reading all files then calculating the statistics?

How about first:

public IEnumerable<String> ReadLinesInDirectory(string path)
{
    return Directory.EnumerateFiles(path)
                    .SelectMany(f => 
                        File.ReadLines(f)
                        .AsEnumerable()
                        .Skip(1));
}

And replace ReadLines with that (ensuring you’re pointing to the right path etc).

---

OK here’s the full explanation, but I think you may need to study C# some more. First, define the ReadLinesInDirectory function I wrote above.

Then replace ProcessMonthlyLogFiles with this instead:

public void ProcessMonthlyLogFiles()
{
    DateTime currentTime = DateTime.Now;

    int month = currentTime.Month - 1;
    int year = currentTime.Year;

    string path = Path.GetDirectoryName(Settings.DailyPath + year + @"\" + month + @"\");

    CalculateMonthlyStatistics(path);
}

And in CalculateMonthlyStatistics have the first three lines (before the GroupBy) as follows:

    public void CalculateMonthlyStatistics(string path)
    {
        var statistics = ReadLinesInDirectory(path)
                         // .GroupBy etc...