Question

I’m trying to implement search in my app. There are two Core Data entities, “Tag” and “DvarTorah”. A tag has just a string. A “DvarTorah” has a title, textual content, and some other properties. I’m trying to figure out the best way to search them quickly. The app ships with about 1200 DvarTorah entities, and even more tags. Right now, I load up an NSFetchedResultsController when my search view controller calls viewDidLoad. Then, when the user types into the search box or changes the scope, I call a method which takes in both the scope bar value and a search term, and filters my array of objects. Here’s how that looks:

- (void) filterArrayWithSearchTerm:(NSString *)searchString andScopeIndex:(NSInteger)scopeIndex{

    if ([searchString isEqualToString:@""]) {
        return;
    }    

    NSMutableArray *unfilteredResults = [[[[self.fetchedResultsController sections] objectAtIndex:0] objects] mutableCopy];

    if (self.filteredArray == nil){
        self.filteredArray = [[[NSMutableArray alloc ] init] autorelease];
    }

    [filteredArray removeAllObjects];

    NSPredicate *predicate = [[[NSPredicate alloc] init] autorelease];

    if (scopeIndex == 0) {
        predicate = [NSPredicate predicateWithFormat:@"dvarTorahTitle CONTAINS[cd] %@", searchString];
    }else if (scopeIndex == 1) {
        predicate = [NSPredicate predicateWithFormat:@"searchableContent CONTAINS[cd] %@", [searchString canonicalString]];            
    }else if (scopeIndex == 2){
        predicate = [NSPredicate predicateWithFormat:@"ANY tags.tagText CONTAINS[cd] %@", searchString];
    }else{
        predicate = [NSPredicate predicateWithFormat:@"(ANY tags.tagText CONTAINS[cd] %@) OR (dvarTorahTitle CONTAINS[cd] %@) OR (searchableContent CONTAINS[cd] %@)", searchString,searchString,searchString];
    }

    for (DvarTorah *dvarTorah in unfilteredResults) {
        if ([predicate evaluateWithObject:dvarTorah]) {
            [self.filteredArray addObject:dvarTorah];
        }
    }

    [unfilteredResults release];
}

The problem is that my search method is terribly slow. I know that CONTAINS is a likely culprit, but even after storing a canonical version of the content (as searchableContent) and attempting to optimize further, the search is horrendously slow. How can I make this faster?

Edit:

Based on Jacob’s initial suggestions, here’s my new method:

    if ([searchString isEqualToString:@""]) {
    return;
}

if (self.filteredArray == nil) {
    self.filteredArray = [[[NSMutableArray alloc ] init] autorelease];
}

[filteredArray removeAllObjects];

NSPredicate *predicate = nil;

if (scopeIndex == 0) {
    predicate = [NSPredicate predicateWithFormat:@"dvarTorahTitle CONTAINS[cd] %@", searchString];
}else if (scopeIndex == 1) {
    predicate = [NSPredicate predicateWithFormat:@"searchableContent CONTAINS[cd] %@", [searchString canonicalString]];            
}else if (scopeIndex == 2){
    predicate = [NSPredicate predicateWithFormat:@"ANY tags.tagText CONTAINS[cd] %@", searchString];
}else{
    predicate = [NSPredicate predicateWithFormat:@"(ANY tags.tagText CONTAINS[cd] %@) OR (dvarTorahTitle CONTAINS[cd] %@) OR (searchableContent CONTAINS[cd] %@)", searchString,searchString,searchString];
}

[self.filteredArray addObjectsFromArray:[[[[[self.fetchedResultsController sections] objectAtIndex:0] objects] mutableCopy] filteredArrayUsingPredicate:predicate]];

}

Edit2:

Not copying the array anymore, still slow:

- (void) filterArrayWithSearchTerm:(NSString *)searchString andScopeIndex:(NSInteger)scopeIndex{

    if ([searchString isEqualToString:@""]) {
        return;
    }

    if (self.filteredArray == nil) {
        self.filteredArray = [[[NSMutableArray alloc ] init] autorelease];
    }

    [filteredArray removeAllObjects];

    NSPredicate *predicate = nil;

    if (scopeIndex == 0) {
        predicate = [NSPredicate predicateWithFormat:@"dvarTorahTitle CONTAINS[cd] %@", searchString];
    }else if (scopeIndex == 1) {
        predicate = [NSPredicate predicateWithFormat:@"searchableContent CONTAINS[cd] %@", [searchString canonicalString]];            
    }else if (scopeIndex == 2){
        predicate = [NSPredicate predicateWithFormat:@"ANY tags.tagText CONTAINS[cd] %@", searchString];
    }else{
        predicate = [NSPredicate predicateWithFormat:@"(ANY tags.tagText CONTAINS[cd] %@) OR (dvarTorahTitle CONTAINS[cd] %@) OR (searchableContent CONTAINS[cd] %@)", searchString,searchString,searchString];
    }

    [self.filteredArray addObjectsFromArray:[[[[self.fetchedResultsController sections] objectAtIndex:0] objects] filteredArrayUsingPredicate:predicate]];
}

Answer 1

There are many things that are chewing up CPU cycles and memory here:

One, you’re making a mutable copy of the results fetched from the NSFetchedResultsController. Why?

Two, you’re using a for..in construct on the result of the above and calling -[NSPredicate evaluateWithObject:] on each. You can revise your predicate search string to work with -[NSArray filteredArrayUsingPredicate:] instead, which is most likely faster than your approach.

Three, there is a rather subtle issue with your predicate variable – you always re-assign it to something else other than the autoreleased empty at the beginning. Give it the default value of nil.

Four, your predicate strings are rather inefficient, as you mentioned.
I think you need to do something called indexing or something like it.

More information on full-text searching with Core Data:

http://developer.apple.com/library/mac/#documentation/cocoa/conceptual/CoreData/Articles/cdPerformance.html

http://cocoawithlove.com/2008/03/testing-core-data-with-very-big.html

http://cocoawithlove.com/2009/11/performance-tests-replacing-core-data.html

http://www.mlsite.net/blog/?page_id=1194

Is SQLite FTS3 still the best way to go for rolling out your full text search?

sqlite Indexing Performance Advice

Full Text Searching in Apple's Core Data Framework