Question

I’m working on an iOS flash-card style learning app that, on load, needs to grab a bunch of data from Core Data. But the data I need is a fairly specific subset of the entity, based on user settings, so there are multiple predicates involved testing equivalence. I’m finding these fetches are super slow and, based on research on SQLite, I think an index would be a good choice here.

Now, I understand (largely from reading other stackoverflow questions) that SQLite and Core Data are two different, basically orthogonal things that should not be confused. But it’s also my understanding that you’re supposed to work through Core Data to do any sort of database work and tweaking; you shouldn’t try to bypass and work directly with SQLite when optimizing or designing object permanence in your app.

But the only thing I can find for indexes in Core Data is that one “indexed” checkbox for each attribute in a model. And that’s just not doing the sort of optimization I’m looking for.

Here’s the fetch request, currently:

NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription entityForName:@"SKUserItem" inManagedObjectContext:context];
fetchRequest.entity = entity;

NSSortDescriptor *sortDescriptor = [[[NSSortDescriptor alloc] initWithKey:@"next" ascending:YES] autorelease];
fetchRequest.sortDescriptors = [NSArray arrayWithObject:sortDescriptor];

NSMutableArray *predicates = [NSMutableArray arrayWithCapacity:6];
[predicates addObject:[NSPredicate predicateWithFormat:@"next < %f", now() + (60.0*60.0*24.0)]];
[predicates addObject:[NSPredicate predicateWithFormat:@"next > %f", nextOffset]];
[predicates addObject:[NSPredicate predicateWithFormat:@"user == %@", user]];
[predicates addObject:[NSPredicate predicateWithFormat:@"langRaw == %d", lang]];

NSArray *stylePredicates = [NSArray arrayWithObjects:[NSPredicate predicateWithFormat:@"styleRaw == %d", SK_SIMP_AND_TRAD], [NSPredicate predicateWithFormat:@"styleRaw == %d", self.style], nil];
[predicates addObject:[NSCompoundPredicate orPredicateWithSubpredicates:stylePredicates]];

if([self.parts count] == 4 || (self.lang == SK_JA && [self.parts count] == 3))
    ;  // don't have to filter by parts; they're studying all of them
else {
    NSMutableArray *partPredicates = [NSMutableArray arrayWithCapacity:[self.parts count]];
    for(NSString *part in self.parts)
        [partPredicates addObject:[NSPredicate predicateWithFormat:@"partRaw == %d", partCode(part)]];
    [predicates addObject:[NSCompoundPredicate orPredicateWithSubpredicates:partPredicates]];
}

NSPredicate *compoundPredicate = [NSCompoundPredicate andPredicateWithSubpredicates:predicates];
fetchRequest.predicate = compoundPredicate;

So essentially what this fetch does is sort by next (the time when the given item is due) and filter for username, language being studied, the style being studied (in Chinese there’s simplified and traditional) and the parts being studied (writing, tone, reading, or definition), and only fetching within a “next” range. Here’s a short list of things I’ve learned from tweaking and fiddling with this:

1. It always scans the whole table, or seems to. Though next is indexed, even if I force it to search a range which I know will return nothing, it still takes several seconds for the fetch to complete.
2. The predicates, any number of predicates, makes this slow. If I remove some but not all, it’s about as slow. If I remove all predicates (thus breaking the app) then it’s much faster.
3. The speed is heavily dependent on how many UserItems there are total in the table. The more items there are, the slower this is. Some people can have tens of thousands of items, and that’s when this fetch can take as much as 10 seconds to complete. This is leading to awkward pauses in my app.
4. The upper bound on the next value was added not because we need it, but because it speeds up the fetch a little bit.
5. Having the query return a subset of the properties in a dictionary (rather than an entire managed object) and fetching the rest lazily is faster, but still not faster enough.

I’m coming from Google App Engine here, so I’m used to the indexes they provide there. Essentially I want that sort of index, but applied to SQLite through Core Data. I found information on adding indexes in SQLite, the kind I would want, but doing this sort of indexing through Core Data, I can’t find any information on that.

Answer 1

What you want is a Compound Index which Core Data supports in iOS 5.0 and later.

You can set it up in Xcode: The Entity inspector has an Indexes section, or if you’re creating the NSEntityDescription in code, use -setCompoundIndexes:.

If you use Xcode, you’d add a line in the Indexes section that says

next,user,langRaw

That way SQL can use an index for your query.