Question

I have 3 animations running in a group: position, scale and rotation

However my rotation animation does not happen smoothly. It happens abruptly in the beginning, while other animations happen smoothly.

Here is my code:

//CABasicAnimation *scaleAnimation
//CABasicAnimation *moveAnimation
CABasicAnimation *rotationAnimation = [CABasicAnimation animationWithKeyPath:@"transform.rotation.y"];
rotationAnimation.autoreverses = NO;  

rotationAnimation.toValue = [NSNumber numberWithFloat: 0];//-(M_PI/3)];
//below line shows the end state, if removed, the layer will assume its initial position after animation, something which I don't want.
[layer setTransform:CATransform3DMakeRotation(0, 0, 1, 0)];    

CAAnimationGroup *theGroup = [CAAnimationGroup animation];
//Code for adding all 3 animations to the group

Edit:

rotationAnimation.toValue = [NSNumber numberWithFloat: -(M_PI/3)];

Removed final transform value from the place of animation declaration, and added it to delegate. (note that in case of animation group, individual animation delegates do not get hit, only the group delegate worked).

Under delegate:

- (void)animationDidStop:(CAAnimation *)theAnimation finished:(BOOL)flag
{   
    NSString * animName = [theAnimation valueForKey:@"animationName"];
    CALayer* layer = [theAnimation valueForKey:@"animationLayer"];

    if ([animName isEqualToString:@"MoveZoomOut"])
    {

       if ([layer.name isEqualToString:@"Layer1"])
          [layer setTransform:CATransform3DMakeRotation(-M_PI/3, 0, 1, 0)];
       else if ([layer.name isEqualToString:@"Layer2"])
          [layer setTransform:CATransform3DMakeRotation(M_PI/3, 0, 1, 0)];
       else
          [layer setTransform:CATransform3DMakeRotation(0, 0, 1, 0)];
    }

    if ([layer respondsToSelector:@selector(setContentsScale:)])
    {
        layer.contentsScale = [UIScreen mainScreen].scale;
    }
}

Answer 1

I have 3 animations running in a group: position, scale and rotation

I suspect that since you are animating both scale and rotation, which is done through the transform property, there might be a clash.

Try to animate scale and rotation in a single animation by concatenating the 2 transforms into one:

CATransform3D t = CATransform3DMakeRotation(0, 0, 1, 0)];
[layer setTransform:CATransform3DScale(t, sx, sy, sz)]; 

EDIT:

I understand that what is happening is pretty unusual, so I will give you a couple of hints as to tricks that in several occasions helped me. I understand, you have 3 layers to animate.

1. wrap each layer animations in [CATransaction begin]/[CATransaction commit];

if that does not help,

1. do not set the final value of the property you are going to animate right after creating the animation itself; rather set it in the animation delegate animationDidStop:finished: method.

In other words, given your code:

CABasicAnimation *rotationAnimation = [CABasicAnimation animationWithKeyPath:@"transform.rotation.y"];
rotationAnimation.autoreverses = NO;  

rotationAnimation.toValue = [NSNumber numberWithFloat: 0];//-(M_PI/3)];
//below line shows the end state, if removed, the layer will assume its initial position after animation, something which I don't want.
[layer setTransform:CATransform3DMakeRotation(0, 0, 1, 0)];    

CAAnimationGroup *theGroup = [CAAnimationGroup animation];

don’t do [layer setTransform:CATransform3DMakeRotation(0, 0, 1, 0)]; there. Rather, do it in:

- (void)animationDidStop:(CAAnimation*)anim finished:(BOOL)flag {
   [layer setTransform:CATransform3DMakeRotation(0, 0, 1, 0)];
}

Since you will have more than one animation going on (and presumably use the same delegate for all of them), you will need a way to tell which animation the animationDidStop: method refers to. To this aim, you could do, e.g.:

//-- se the delegate
rotationAnimation.delegate = self;
[rotationAnimation setValue:layer forKey:@"animationLayer"];

and in the delegate method:

- (void)animationDidStop:(CAAnimation*)anim finished:(BOOL)flag {

   [CATransaction begin];
   [CATransaction setDisableActions:YES];

   CALayer* layer = [anim valueForKey:@"animationLayer"];
   layer.transform = ...;

   [CATransaction commit];

}

I know that this seems a bit contrived. And it is. But it is the only way I could fix an issue similar to what you are reporting.

Hope this helps.

EDIT:

For the glitch when using animationDidStop:, try to wrap animationDidStop: content in:

[CATransaction begin];
[CATransaction setDisableActions:YES];
...
[CATransaction commit];