Question

I have written an iPhone and iPad app that should streams e.g. 4 videos over HTTP. Each video must be requested as the previous one starts, so I cannot ask for all of them at the start. The exception to this is that I can request the second video (let’s call it the content) at the start. Let’s call the other videos ads.

I have a URL for each ad or video which returns some XML description containing the URL of the actual content. (Actually it’s JSON in the case of the main video, but never mind.)

When the XML returns, I find the content URL and make an AVPlayerItem for it. When the first ad returns, I make the AVQueuePlayer with it. If the main video has already returned by then, I make an AVPlayerItem for it, and insert it in the queue player after the first ad.

There are two formats available for both ad and video, MP4 and 3GP. I choose 3GP if the device is not on wifi, or is a low-end device (iPhone 3G and iPad 2nd gen are examples.)

I then observe various things on the player and any items I have created. First the current item changing:

-(void)playerCurrentItemChanged:(NSString*)aPath ofPlayer:(AVQueuePlayer*)aQueuePlayer change:(NSDictionary*)aChange {

if ([aPath isEqualToString:kCurrentItemKey]) {

    if (!_quitting) {

        [self performSelectorOnMainThread:@selector(queuePlayerCurrentItemChanged) withObject:nil waitUntilDone:NO];
     }
  }
}

Note I call another method on the main thread, because it says in the docs that non-atomic properties of the AVPlayer should be used this way.

This method looks like this:

-(void)queuePlayerCurrentItemChanged {

NSAssert([NSThread isMainThread], @"FIX ME! VideoController method called using a thread other than main!");

AVPlayerItem* playerItem = _queuePlayer.currentItem;

if (playerItem) {

    VideoItem* videoItem = [self findVideoItemFromPlayerItem:playerItem];
    [self getReadyToPlay:videoItem];

    // don't continue to FF if it's an ad
    if (videoItem.isAd && (_queuePlayer.rate > 1.0)) {

        _queuePlayer.rate = 1.0;
    }
}
else {
    NSLog(@"queuePlayerCurrentItemChanged to nil!!!");
}
}

A VideoItem is my own class wrapping an AVPlayerItem, and adding additional data that I need to track.

Basically, getReadyToPlay sets up the UI, according to whether it’s an ad (not allowed to FF) or the main video. The code also stops FFing if we’re transitioning to an ad.

I also observe the status of any item that has been created. The method that follows is similarly called on the main thread, if the status has become playable for any item:

-(void)queuePlayerItemStatusPlayable:(AVPlayerItem*)aPlayerItem {

NSAssert([NSThread isMainThread], @"FIX ME! VideoController method called using a thread other than main!");

VideoItem* videoItem = [self findVideoItemFromPlayerItem:aPlayerItem];
NSLog(@"queuePlayerItemStatusPlayable for item %@",videoItem);

if (_queuePlayer.currentItem != aPlayerItem) {
    NSLog(@"  but playable status is for non current item %@, ignoring",videoItem);
    return;
}

if (_videoItemIndex == 0) {
    NSLog(@"   and playable status is for item 0 - call getReadyToPlay");
    [self getReadyToPlay:videoItem]; // the first item doesn't get a current item notification so do this here
}

[self playIfReady]; //pausenotadvance (do this every time now)

}

If it’s the first ad, I have to call getReadyToPlay, as there’s never a current item notification for the player for this. I used to call playIfReady just for this item too, but now I call it for any current item, in an attempt to avoid stalling.

Similarly, this code is called if an item gets a status of AVPlayerItemStatusFailed:

-(void)queuePlayerItemStatusFailed:(AVPlayerItem*)aPlayerItem {

NSAssert([NSThread isMainThread], @"FIX ME! VideoController method called using a thread other than main!");

VideoItem* videoItem = [self findVideoItemFromPlayerItem:aPlayerItem];
if (videoItem.isAd) {

    // this seems to be the only notification that we've run out of video when reachability is off
    if (appDelegate._networkStatus == NotReachable) {
        NSLog(@"AVPlayerItemStatusFailed when not reachable, show low bandwidth UI");
        if (!_isPaused && !_lowBandwidthUIShowing) {

            [_queuePlayer pause];;
            [self showLowBandwidthUI];
        }
        return;
    }

    NSError* error = aPlayerItem.error;

    if (aPlayerItem == _queuePlayer.currentItem) {

        NSLog(@"AVPlayerStatusFailed playing currently playing ad with error: %@",error.localizedDescription);
        if (videoItem.isLast) {
            NSLog(@"Failed to play last ad, quitting");
            [self playEnded];
        }
        else {
            NSLog(@"Not the last ad, advance");
            [_queuePlayer advanceToNextItem]; 
        }
    }
    else {

        NSLog(@"Error - AVPlayerStatusFailed on non-playing ad with error: %@",error.localizedDescription);
        [_queuePlayer removeItem:aPlayerItem];
     }
}
else {
    // This is can be an invalid URL in the main video JSON or really bad network
    // Assuming invalid URLS are pretty rare by the time we're in the app store, blame the network
    // Whatever, give up because it's the main video

    NSError* error = aPlayerItem.error;
    if (appDelegate._networkStatus == ReachableViaWiFi) {

        if (!_alertShowing) {
            NSLog(@"Error - AVPlayerStatusFailed playing main video, bad JSON? : error %@",error.localizedDescription);
            [self showServerAlertAndExit];
        }
    }
    else {
        if (!_alertShowing) {
            NSLog(@"Error - AVPlayerStatusFailed playing main video, bandwidth? : error %@",error.localizedDescription);
             [self showNetworkAlertAndExit];                            
        }
    }
}
return;

}

When a further ad comes in, I either add it to the end of the player or replace the current item. I only do the latter if the current item is nil, which means the player has finished the current piece of video and stalled waiting for more:

-(void) addNewItemToVideoPlayer:(AVPlayerItem*)aPlayerItem {

NSAssert([NSThread isMainThread], @"FIX ME! VideoController method called using a thread other than main!");

if (_queuePlayer.currentItem == nil) {
    NSLog(@"    CCV replaced nil current item, player %@",_queuePlayer);
    [_queuePlayer replaceCurrentItemWithPlayerItem:aPlayerItem];
    if (!_isPaused)
        [_queuePlayer play];
}
else if ([_queuePlayer canInsertItem:aPlayerItem afterItem:((VideoItem*)[_videoItems objectAtIndex:_indexLastCued]).avPlayerItem]) {
    NSLog(@"    CCV inserted item after valid current item, player %@",_queuePlayer);
    [_queuePlayer insertItem:aPlayerItem afterItem:((VideoItem*)[_videoItems objectAtIndex:_indexLastCued]).avPlayerItem];
}
}

This code seems to work pretty well on the simulator/wifi, and possibly high-end devices.

An iPhone 3G on a slowish 3G network shows a variety of not very repeatable defects.

I do observe playback likely to keep up, and call this on the main thread:

-(void)queuePlayerLikelyToKeepUp:(AVPlayerItem*)aPlayerItem {

NSAssert([NSThread isMainThread], @"FIX ME! VideoController method called using a thread other than main!");

if (aPlayerItem.playbackLikelyToKeepUp == NO) {

    if (!_isPaused) {

        [_queuePlayer pause];
        [self showLowBandwidthUI];
        [self performSelector:@selector(restartVideo) withObject:nil afterDelay:1.0]; // delay and try again
    }
}
else {
    // if we forced the showing of UI due to low bandwidth, fade it out, remove spinner
    if (_lowBandwidthUIShowing) {
        [self hideLowBandwidthUI];
        [_queuePlayer play];
    }
}
}

This basically pauses the video and shows the UI (which shows a progress bar showing how much video has been downloaded, to give the user feedback as to why the video has stalled). It then tries to restart the video a second later.

The worst errors I see on the iPhone 3G are to do with total stalling. Sometimes I get an AVPlayerItemStatusFailed for the main video (the error message is unhelpful – just “unknown error”- AVFoundationErrorDomain error -11800) so the user exits the stalled video and tries again, but it then seems that once the player is in this state, it won’t play any videos, even ones it has played before. And yet this is a totally new AVQueuePlayer – I exit my VideoController after each set of videos is done, or when the user gets fed up with them.

Videos can also get into a “no ads” mode. The main video plays, but then no ads show, and for all subsequent videos, no ads show.

I’m aware this is a difficult if not impossible question for you to answer, but I’ve been asked to ask it, so I’m going ahead.

Can you see anything I’m doing wrong?

More generally, can you give a brief tutorial on how to use AVQueuePlayer in a situation like this, where all the video URLS are not known when the AVQueuePlayer is created? What are the does and don’ts of ABQueuePlayer and AVPlayer?

Can you give advice on dealing with low-bandwidth situations with AVQueuePlayer? What causes it to stall in an un-restartable way?

Have you any thoughts on what has to be done on the main thread, and what doesn’t?

Answer 1

The answer was that all methods (not just properties) of the AVQueuePlayer must be called on the main thread. Also, KVO observing must be started and stopped on the main thread.

If you don’t do this, in even one tiny case, your player will eventually get into a state where it refuses to play. And worse, the mediaserverd process backing the AVQueuePlayer is also in such a state, so that even starting to play a new video, and recreating the AVQueuePlayer, will not work.