Question

We have a large project that needs to sync large files from a server into a ‘Library’ in the background. I read subclassing NSOperation is the most flexible way of multithreading iOS tasks, and attempted that. So the function receives a list of URLs to download & save, initialises an instance of the same NSOperation class and adds each to an NSOperation queue (which should download only 1 file at a time).

-(void) LibSyncOperation {    
    // Initialize download list. Download the homepage of some popular websites
    downloadArray = [[NSArray alloc] initWithObjects:@"www.google.com",
                                                     @"www.stackoverflow.com",
                                                     @"www.reddit.com",
                                                     @"www.facebook.com", nil];

    operationQueue = [[[NSOperationQueue alloc]init]autorelease];
    [operationQueue setMaxConcurrentOperationCount:1]; // Only download 1 file at a time
    [operationQueue waitUntilAllOperationsAreFinished];

    for (int i = 0; i < [downloadArray count]; i++) {
        LibSyncOperation *libSyncOperation = [[[LibSyncOperation alloc] initWithURL:[downloadArray objectAtIndex:i]]autorelease];
        [operationQueue addOperation:libSyncOperation];
    }
}

Now, those class instances all get created fine, and are all added to the NSOperationQueue and begin executing. BUT the issue is when it’s time to start downloading, the first file never begins downloading (using an NSURLConnection with delegate methods). I’ve used the runLoop trick I saw in another thread which should allow the operation to keep running until the download is finished. The NSURLConnection is established, but it never starts appending data to the NSMutableData object!

@synthesize downloadURL, downloadData, downloadPath;
@synthesize downloadDone, executing, finished;

/* Function to initialize the NSOperation with the URL to download */
- (id)initWithURL:(NSString *)downloadString {

    if (![super init]) return nil;

    // Construct the URL to be downloaded
    downloadURL = [[[NSURL alloc]initWithString:downloadString]autorelease];
    downloadData = [[[NSMutableData alloc] init] autorelease];

    NSLog(@"downloadURL: %@",[downloadURL path]);

    // Create the download path
    downloadPath = [NSString stringWithFormat:@"%@.txt",downloadString];
    return self;
}

-(void)dealloc {
    [super dealloc];
}

-(void)main {

    // Create ARC pool instance for this thread.   
    // NSAutoreleasePool *pool = [[NSAutoreleasePool alloc]init]; //--> COMMENTED OUT, MAY BE PART OF ISSUE

    if (![self isCancelled]) {

        [self willChangeValueForKey:@"isExecuting"];
        executing = YES;

        NSURLRequest *downloadRequest = [NSURLRequest requestWithURL:downloadURL];
        NSLog(@"%s: downloadRequest: %@",__FUNCTION__,downloadURL);
        NSURLConnection *downloadConnection = [[NSURLConnection alloc] initWithRequest:downloadRequest delegate:self startImmediately:NO];

        // This block SHOULD keep the NSOperation from releasing before the download has been finished
        if (downloadConnection) {
            NSLog(@"connection established!");
            do {
                [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];
            } while (!downloadDone);

        } else {
            NSLog(@"couldn't establish connection for: %@", downloadURL);

            // Cleanup Operation so next one (if any) can run
            [self terminateOperation];
        }
            }
    else { // Operation has been cancelled, clean up
        [self terminateOperation];
    }

// Release the ARC pool to clean out this thread 
//[pool release];   //--> COMMENTED OUT, MAY BE PART OF ISSUE
}

#pragma mark -
#pragma mark NSURLConnection Delegate methods
// NSURLConnectionDelegate method: handle the initial connection 
-(void)connection:(NSURLConnection *)connection didReceiveResponse:(NSHTTPURLResponse*)response {
    NSLog(@"%s: Received response!", __FUNCTION__);
}

// NSURLConnectionDelegate method: handle data being received during connection
-(void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
    [downloadData appendData:data];
    NSLog(@"downloaded %d bytes", [data length]);
}

// NSURLConnectionDelegate method: What to do once request is completed
-(void)connectionDidFinishLoading:(NSURLConnection *)connection {
    NSLog(@"%s: Download finished! File: %@", __FUNCTION__, downloadURL);
    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *docDir = [paths objectAtIndex:0];
    NSString *targetPath = [docDir stringByAppendingPathComponent:downloadPath];
    BOOL isDir; 

    // If target folder path doesn't exist, create it 
    if (![fileManager fileExistsAtPath:[targetPath stringByDeletingLastPathComponent] isDirectory:&isDir]) {
        NSError *makeDirError = nil;
        [fileManager createDirectoryAtPath:[targetPath stringByDeletingLastPathComponent] withIntermediateDirectories:YES attributes:nil error:&makeDirError];
        if (makeDirError != nil) {
            NSLog(@"MAKE DIR ERROR: %@", [makeDirError description]);
            [self terminateOperation];
        }
    }

    NSError *saveError = nil;
    //NSLog(@"downloadData: %@",downloadData);
    [downloadData writeToFile:targetPath options:NSDataWritingAtomic error:&saveError];
    if (saveError != nil) {
        NSLog(@"Download save failed! Error: %@", [saveError description]);
        [self terminateOperation];
    }
    else {
        NSLog(@"file has been saved!: %@", targetPath);
    }
    downloadDone = true;
}

// NSURLConnectionDelegate method: Handle the connection failing 
-(void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error {
    NSLog(@"%s: File download failed! Error: %@", __FUNCTION__, [error description]);
    [self terminateOperation];
}

// Function to clean up the variables and mark Operation as finished
-(void) terminateOperation {
    [self willChangeValueForKey:@"isFinished"];
    [self willChangeValueForKey:@"isExecuting"];
    finished = YES;
    executing = NO;
    downloadDone = YES;
    [self didChangeValueForKey:@"isExecuting"];
    [self didChangeValueForKey:@"isFinished"];
}

#pragma mark -
#pragma mark NSOperation state Delegate methods
// NSOperation state methods
- (BOOL)isConcurrent {
    return YES;
}
- (BOOL)isExecuting {
    return executing;
}
- (BOOL)isFinished {
    return finished;
}

NOTE: If that was too unreadable, I set up a QUICK GITHUB PROJECT HERE you can look through. Please note I’m not expecting anyone to do my work for me, simply looking for an answer to my problem!

I suspect it has something to do with retaining/releasing class variables, but I can’t be sure of that since I thought instantiating a class would give each instance its own set of class variables. I’ve tried everything and I can’t find the answer, any help/suggestions would be much appreciated!

UPDATE: As per my answer below, I solved this problem a while ago and updated the GitHub project with the working code. Hopefully if you’ve come here looking for the same thing it helps!

Answer 1

In the interests of good community practice and helping anyone else who might end up here with the same problem, I did end up solving this issue and have updated the GitHub sample project here that now works correctly, even for multiple concurrent NSOperations!

It’s best to look through the GitHub code since I made a large amount of changes, but the key fix I had to make to get it working was:

[downloadConnection scheduleInRunLoop:[NSRunLoop mainRunLoop] forMode:NSDefaultRunLoopMode];

This is called after the NSURLConnection is initialized, and just before it is started. It attaches the execution of the connection to the current main run loop so that the NSOperation won’t prematurely terminate before the download is finished. I’d love to give credit to wherever first posted this clever fix, but it’s been so long I’ve forgotten where, apologies. Hope this helps someone!