Facts:
- I have a desktop app backed by a SQL CE database.
- Many of the tables are read-only.
- I’m using EF Code First to generate the database and handle crud operations.
- I generally keep one DbContext open for the whole session and dispose on app exit. (Note: I’ve tried creating a new DbContext more frequently and found that performance was actually worse. Also, I don’t ever need to roll back changes.)
- For several of my entities, I have added some getters to provide necessary data derived from other properties of the entity. A few of these getters actually perform LINQ queries (which I suspect is part of my problem).
Everything is working, but I’m realizing now that I need to do some optimization. Performance is somewhat sluggish in general, but I’m especially concerned about one really giant query users need to run periodically that flattens pretty much the whole database into a single table then outputs it to a delimited text file. This query is taking much longer than I’d like.
One idea I have for speeding things up would be to store anything that is read-only in a List instead of grabbing it from my DbContext each time I need it. The tricky part is that, while I could easily grab all my Car entities from the DbContext and store them in a list, how would I ensure that all the “helper properties”, such as public virtual ICollection<Wheel> Wheels { get; set; } or public virtual Engine Engine { get; set; } get stored as well? It doesn’t do a lot of good to iterate through all the Car entities if all the associated entities are not iterated as well.
The other idea I have for improving performance is to come up with a way to cache the getters that are the most processor intensive. As I mentioned above, some of my getters actually perform LINQ queries. I know this is sapping performance, but since some of these are dependent on data that can change, caching becomes quite tricky. I have a feeling it would require implementing INotifyPropertyChanged.
What I’d like is a strategy for improving performance without throwing away all the good things that EF is already doing for me. There’s no point in me storing something in a list, for example, if EF has already iterated the list and will not iterate the list again unless and until it becomes out of date.
Given my specific situation and the above background, what strategy(ies) do you recommend to speed up my app?
Clarification
As I wrote in comments below…
For the large query, I’m iterating through all my entities (putting each into a List<TEntity>), then doing a giant join, then selecting the actual data I need. For this final selection, some of the data is raw (a basic property of the entity), while some is calculated (a property getter that depends on basic properties of the entity). For the calculated properties, there are sometimes LINQ queries involved. So, a lot of the heavy lifting is done in C# rather than via the database (which is file-based, being SQL CE).
For everything else in my app. I directly query the database when possible and make no attempt to cache anything. If I could figure out a way to completely cache everything that is read-only into memory, I have to believe that would improve performance, but I’m not entirely clear on how to do that given that (1) I have entities that store other entities and (2) the entities have getters, some of which perform queries.
Caching is your friend: http://support.microsoft.com/kb/323290