Question

Is there a way to obtain the matrix which rotates a plane to a new orientation, given its new normal vector

The following image depicts what is described

Answer 1

Given the old normal N and the new normal N' you can obtain the rotation by:

RotationAxis = cross(N, N')
RotationAngle = arccos(dot(N, N') / (|N| * |N'|))

Where

• cross(x, y) is the cross product of the vectors x and y
• dot(x, y) is the dot product of the vectors x and y
• |x| is the length of the vector x

This will rotate the old normal onto the new one by the shortest way possible.

Notes

• RotationAngle will be in radians (if arccos returns radians as it does in most implementations)
• arccos is the inverse of the cosine function. It is necessary because dot(N, N') = |N| * |N'| * cos(RotationAngle) where RotationAngle is the angle between the vectors.
• RotationAxis is not normalized
• If both normals are normalized the division by (|N| * |N'|) becomes unnecessary (in fact if N is normalized you can leave out |N| of the product and if N' is normalized then leave out |N'|)
• This method will fail if N' = -N (as there are infinite many shortest ways)

How does it work?

The first observation is that the two normals will always define (at least) one plane in which both are lying. The smallest angle that parts them will be measured inside this plane too.

So the RotationAxis vector will be the normal of the plane that encloses both N and N' and the RotationAngle is the smallest angle between the two mentioned earlier.

So by rotating around RotationAxis by the RotationAngle the old normal N is rotated inside the plane, on the shortest path towards N'.