Home/ Questions/Q 7964205
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-04T05:49:39+00:00

I am having usues with performance when inserting 2000 records in my Core Data

  • 0

I am having usues with performance when inserting 2000 records in my Core Data application..

The data is being downloaded from a server as a big JSON file, and parsed into a JSON dictionary, all this is well, and the time to passe the dictionary is nothing…

The issue is that each insert into my database takes longer and longer ?

During my import, i save the context for each 100 docs to keep the memory down, and the first document takes 0.005434ms to save, the last takes 0.039297ms to save..

I am doing all this import in a seperate thread with a completely new ManagedContext with undomanager set to nil..

This is the loop that runs through all the docs in the dictionary

    NSArray *docs = [docsData objectForKey:@"docs"];
    for(NSDictionary *doc in docs){
        if(counter++ % 100){
            [context save:nil];
        }
        NSDate *start = [NSDate date];
        [Document documentWithDictionary:doc lastModifiedDate:[NSDate date] inLevels:nil inManagedObjectContext:context];
        NSDate *end = [NSDate date];

        NSLog(@"time used pr doc = %f",[end timeIntervalSinceDate:start]);
    }
    [context save:nil];

And here is the code that inserts the doc

     NSFetchRequest *req = [NSFetchRequest fetchRequestWithEntityName:@"Document"];
    req.predicate = [NSPredicate predicateWithFormat:@"id = %@", [data valueForKey:@"id"]];
    NSArray *matches = [context executeFetchRequest:req error:&error];

    if(matches){    
        if([matches count]){
            document = [matches lastObject];
        }else {
            document = [NSEntityDescription insertNewObjectForEntityForName:@"Document" inManagedObjectContext:context];
        }
   }

Can someone shed any light as to why the inserts takes longer and longer?

In the simulator the used time pr doc is pretty much constant, but on the phone it is not ?

This might be a minor issue, but since i can have anywhere from 2000 to 30000 records in the database, this actually becomes a factor with large imports..

Thanks a lot 🙂

/Jacob

UPDATE —–

After doing ONLY insertions in the database, that is, with no fetch for existing records, these are the times..

WITH Fetch:

1100 docs – 54.6s

2349 docs – 194.9s

1872 docs – 222.1s

WITHOUT Fetch.

1100 docs – 34.4s

2349 docs – 74.19s

1872 docs – 59.1s

So, the conclution is that it is my fetch request with the increasing amount of docs that is taking longer and longer.. but that also makes sense 🙂 don’t know why i did’nt think of that earlier… So now the solution is to check if the sync is the first one, and then import documents without the fetch for any existing document.

/Jacob

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Based on your code, the issue has nothing to do with inserting anything. Nothing actually gets persisted to the database until you call the save method. I’m assuming the “…code that inserts the doc” is the code in the documentsWithDictionary:lastModifiedDate:inLevels:inManagedObjectContext: method. You are not actually inserting anything here, but creating a new ManagedObject in memory. However, you ARE querying the database every time you do this. As the number of records in the database grows, the query could take slightly longer to find a record of the given id.

    Apple has outlined some good practices for efficiently importing large data sets: http://developer.apple.com/library/mac/#documentation/cocoa/conceptual/coredata/Articles/cdImporting.html#//apple_ref/doc/uid/TP40003174-SW1

    In particular you will want to read “Implement Find-or-Create Efficiently”. Following their guidelines you can limit your database reads to one per batch of records, or simply once for the entire data set you’re importing.

    • 0