Question

I’ve been learning C# / LINQ / ASP.NET / MVC 3 / EF for a few months now comming from Java / Icefaces / Ibatis base (Real world uses .NET D;). I really enjoy LINQ / Entity Framework from the .NET Framework but I’m having a few issues understand what’s really happening behind the scenes.

Here’s my problem:

I’m using a AJAX / JSON fed jQuery datatable (that I highly recommend to anyone in need of a free web datatable system by the way). I have a method in my MVC3 application that returns a JSON result of the data needed by the table, doing the sorting and all. Everything is working nicely and smoothly. However, I’m having a concern with the "dirty" hack I had to do to make this work.

Here’s the complete code:

//inEntities is the Entity Framework Database Context
//It includes the following entities:
//  Poincon
//  Horaire
//  HoraireDetail
//Poincon, Horaire and HoraireDetail are "decorated" using the Metadata technic which
//adds properties methods and such to the Entity (Like getEmploye which you will see in
//the following snippet)
//
//The Entity Employe is not a database data and therefor not handled by the EF.
//Instead, it is a simple object with properties that applies Lazy Loading to get an
//Employe Name based off of his Employe ID in the Active Directory. An employe object
//can be constructed with his Employe ID which will expose the possibility of getting
//the Employe Name from the AD if needed.

[HttpPost]
public JsonResult List(FormCollection form)
{
    String sEcho;
    int iDisplayStart;
    int iDisplayLength;
    String sSearch;
    int iSortingCols;
    Dictionary<String, String> sorting;

    try
    {
        sEcho = form["sEcho"];
        iDisplayStart = int.Parse(form["iDisplayStart"]);
        iDisplayLength = int.Parse(form["iDisplayLength"]);
        sSearch = form["sSearch"];
        iSortingCols = int.Parse(form["iSortingCols"]);

        sorting = new Dictionary<string,string>();
        for (int i = 0; i < iSortingCols; i++)
            sorting.Add(form["mDataProp_" + form["iSortCol_" + i]].ToUpper(), form["sSortDir_" + i].ToUpper());
    }
    catch
    {
        HttpContext.Response.StatusCode = 500;
        return null;
    }

    var qPoincon = inEntities.Poincons.AsEnumerable();
    var lPoincon = qPoincon.Select(o => new
    {
        o.id,
        emp = o.getEmploye(),
        o.poinconStart,
        o.poinconEnd,
        o.commentaire,
        o.codeExceptions
    }).AsEnumerable();

    //Search
    lPoincon = lPoincon.Where(p => (p.emp.empNoStr.Contains(sSearch) || p.emp.empNom.Contains(sSearch) || (p.commentaire != null && p.commentaire.Contains(sSearch))));

    //Keep count
    int iTotalDisplayRecords = lPoincon.Count();

    //Sorting
    foreach(KeyValuePair<String,String> col in sorting)
    {
        switch (col.Key)
        {
            case "EMPNO":
                if (col.Value == "ASC")
                    lPoincon = lPoincon.OrderBy(h => h.emp.empNo);
                else
                    lPoincon = lPoincon.OrderByDescending(h => h.emp.empNo);
                break;
            case "POINCONSTART":
                if (col.Value == "ASC")
                    lPoincon = lPoincon.OrderBy(h => h.poinconStart);
                else
                    lPoincon = lPoincon.OrderByDescending(h => h.poinconStart);
                break;
            case "POINCONEND":
                if (col.Value == "ASC")
                    lPoincon = lPoincon.OrderBy(h => h.poinconEnd);
                else
                    lPoincon = lPoincon.OrderByDescending(h => h.poinconEnd);
                break;
            case "COMMENTAIRE":
                if (col.Value == "ASC")
                    lPoincon = lPoincon.OrderBy(h => h.commentaire);
                else
                    lPoincon = lPoincon.OrderByDescending(h => h.commentaire);
                break;
        }
    }

    //Paging
    lPoincon = lPoincon.Skip(iDisplayStart).Take(iDisplayLength);

    //Building Response
    var jdt = new
    {
        iTotalDisplayRecords = iTotalDisplayRecords,
        iTotalRecords = inEntities.Poincons.Count(),
        sEcho = sEcho,
        aaData = lPoincon
    };
    return Json(jdt);
}

As you can see, when I’m grabbing the entire list of "Poincons" from the EF and turning it into a Enumerable. From my current understanding, turning the LINQ query into a Enumerable "kills" the link to the EF, or in other words, will generate the SQL required to get that list at that point instead of keeping the LINQ data until the end and execute a percise query that will return only the data you require. After turning this LINQ Query into a Enumerable, I’m heavily filtering the LINQ (since there is paging, sorting, searching in the datatable). This leads me to thinkg that what my code is currently doing is "Grab all the "Poincons" from the database and put it into the web server’s memory as a Enumerable, do your work with the Enumerable then serialize the result as a JSON string and send it to the client.

If I’m correct, the performance hit is quite heavy when you hit the couple thousand of entries (which will happen quite fast once in production… everytime an employe comes to work, it will add 1 entry. 100 employes, ~300 work days a year, you get the idea).

The reason for this hack is that the EF does not know what "getEmploye" method of "Poincon" is, therefor throwing an exception at runtime similar to this:

LINQ to Entities ne reconnaît pas la méthode « PortailNorclair.Models.Employe getEmploye() », et cette dernière ne peut pas être traduite en expression de magasin.

Approximated traduction (If anyone can let me know in a comment how to configure IIS / ASP.NET to display errors in english while keeping the globalization in a foreign language, I would be really grateful. French information about error messages is sometimes lacking):

LINQ to Entity does not recognize the method " PortailNorclair.Models.Employe getEmploye()" and the following could not be translated to a SQL expression.

The "getEmploye" method instances and returns a Employe object with the employe id found in the Poincon object. That Employe object has properties that "lazy loads" information like the employe name from the Active Directory.

So the question is: How can I avoid the performance hit from using .AsEnumerable() on the non-filtered list of objects?

Thanks a lot!

Answer 1

The “getEmploye” method instances and returns a Employe object with
the employe id found in the Poincon object. That Employe object has
properties that “lazy loads” information like the employe name from
the Active Directory.

You should be storing the Employee Name in the database, so you can then order, sort, skip and take in your Linq Query without having to load every employee object.

If empNoStr, empNom, and empNo were all in the database, you could retrieve just the records you want, and call getEmploye() (loading whatever else you need from active directory, or wherever) for each of those.