Question

I have the following query:

if (idUO > 0)
{
    query = query.Where(b => b.Product.Center.UO.Id == idUO);
}
else if (dependencyId > 0)
{
    query = query.Where(b => b.DependencyId == dependencyId );
}
else
{
    var dependencyIds = dependencies.Select(d => d.Id).ToList();

    query = query.Where(b => dependencyIds.Contains(b.DependencyId.Value));
}

[...] <- Other filters...

if (specialDateId != 0)
{
    query = query.Where(b => b.SpecialDateId == specialDateId);
}

So, I have other filters in this query, but at the end, I process the query in the database with:

return query.OrderBy(b => b.Date).Skip(20 * page).Take(20).ToList(); // the returned object is a Ticket object, that has 23 properties, 5 of them are relationships (FKs) and i fill 3 of these relationships with lazy loading

When I access the first page, its OK, the query takes less than one 1 second, but when I try to access the page 30000, the query takes more than 20 seconds. There is a way in the linq query, that I can improve the performance of the query? Or only in the database level? And in the database level, for this kind of query, which is the best way to improve the performance?

Answer 1

I think you’ll find that the bottleneck is occurring at the database. Here’s why;

query.

You have your query, and the criteria. It goes to the database with a pretty ugly, but not too terrible select statement.

.OrderBy(b => b.Date)

Now you’re ordering this giant recordset by date, which probably isn’t a terrible hit because it’s (hopefully) indexed on that field, but that does mean the entire set is going to be brought into memory and sorted before any skipping or taking occurs.

.Skip(20 * page).Take(20)

Ok, here’s where it gets rough for the poor database. Entity is pretty awful at this sort of thing for large recordsets. I dare you to open sql profiler and view the random mess of sql it’s sending over.

When you start skipping and taking, Entity usually sends queries that coerce the database into scanning the entire giant recordset until it finds what you are looking for. If that’s the first ordered records in the recordset, say page 1, it might not take terribly long. By the time you’re picking out page 30,000 it could be scanning a lot of data due to the way Entity has prepared your statement.

I highly recommend you take a look at the following link. I know it says 2005, but it’s applicable to 2008 as well.

http://www.codeguru.com/csharp/.net/net_data/article.php/c19611/Paging-in-SQL-Server-2005.htm

Once you’ve read that link, you might want to consider how you can create a stored procedure to accomplish what you’re going for. It will be more lightweight, have cached execution plans, and is pretty well guaranteed to return the data much faster for you.

Barring that, if you want to stick with LINQ, read up on Compiled Queries and make sure you’re setting MergeOption.NoTracking for read-only operations. You should also try returning an Object Query with explicit Joins instead of an IQueryable with deferred loading, especially if you’re iterating through the results and joining to other tables. Deferred Loading can be a real performance killer.