Question

I have a table with an event id pk, date column and an event type column.
I want to use linq to get the count of events in each day.
The issue is that the table is sparse, i.e. values are not stored in days which did not have any events.

Since I want to use this data for a line chart, I need to fill out the data with the missing dates and give them a value of zero.

Is these any way to do this inside linq? or do I have to do this manually?

Is there any recommended method of doing this?

Edit:

I created the following method:

public string GetDailyData(int month, int year)
    {
        int days = DateTime.DaysInMonth(year,month);
        DateTime firstOfTheMonth = new DateTime(year, month, 1);
        PaymentModelDataContext db = new PaymentModelDataContext();
        var q = from daynumber in Enumerable.Range(0, days)
                let day = firstOfTheMonth.AddDays(daynumber)
                join data in db.TrackingEvents on day equals data.timestamp.Day into d2
                from x in d2.DefaultIfEmpty()
                select Tuple.Create(x.Key, x.Value);

        return ParseJson(q);
    }

The problem is I get an error on the ‘join’ keyword:
“The type of one of the expressions in the join clause is incorrect. Type inference failed in the call to ‘GroupJoin'”

Edit 2:
I made the changes suggested and tried to group the results.
When I send them to the parsing function, I get a null object ref error.
Here is the new code:

[WebMethod]
    public string GetDailyData(int month, int year)
    {
        int days = DateTime.DaysInMonth(year, month);
        DateTime firstOfTheMonth = new DateTime(year, month, 1);
        PaymentModelDataContext db = new PaymentModelDataContext();
        var q = from daynumber in Enumerable.Range(0, days)
                let day = firstOfTheMonth.AddDays(daynumber)
                join data in db.TrackingEvents on day equals data.timestamp.Date into d2
                from x in d2.DefaultIfEmpty()
                group x by x.timestamp.Date;

        return ParseJson(q);
    }

And the parsing function:

private string ParseJson<TKey, TValue>(IEnumerable<IGrouping<TKey, TValue>> q)
    {
        string returnJSON = "[{ \"type\" : \"pie\", \"name\" : \"Campaigns\", \"data\" : [ ";
        foreach (var grp in q)
        {
            double currCount = grp.Count();
            if (grp.Key != null)
                returnJSON += "['" + grp.Key + "', " + currCount + "],";
            else
                returnJSON += "['none', " + currCount + "],";
        }
        returnJSON = returnJSON.Substring(0, returnJSON.Length - 1);
        returnJSON += "]}]";

        return returnJSON;
    }

Answer 1

You should be able to use LINQ. One method is to use Enumerable.Range to create a collection of dates between the min and max dates, and then perform an outer join (using GroupJoin) against the sparse table. (See MSDN Reference: How to Perform Outer Joins (C# Programming Guide))

For instance, if numdays is the date range (in days), MinDate is the initial date, and SparseData is your sparse data, and SparseData has an instance property Day that specifies the date, then you might do:

var q = Enumerable.Range(0, numdays)
        .Select(a => MinDate.AddDays(a))
        .GroupJoin(
           SparseData,
           q=>q,
           sd=>sd.Day,
           (key, value) =>
               Tuple.Create(
                     key,
                     value.DefaultIfEmpty().First()
               )
          );

Or, equivalently,

  var q2 = from daynumber in Enumerable.Range(0, numdays)
           let day = MinDate.AddDays(daynumber)
           join data in SparseData on day equals data.Day into d2
           from x in d2.DefaultIfEmpty()
           select Tuple.Create(x.Key, x.Value);