Question

How do I calibrate an xna camera projection so that kinect’s skeleton data points line up with the video feed when overlayed?

Answer 1

I’ve done this in WPF, I assume it will be the same with XNA..

This is the code I used, which I believe was taken from one of the sample Kinect projects in the SDK’s documentation. Your results may vary. Note that the 320×240 part seems to have nothing to do with the actual resolution you’re using.. I had the depth stream set to 640×480 and the RGB stream set to 1280×1024.

nui is my Runtime object.

private Point getDisplayPosition(Joint joint)
{
    float depthX, depthY;
    nui.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
    depthX = Math.Max(0, Math.Min(depthX * 320, 320));  //convert to 320, 240 space
    depthY = Math.Max(0, Math.Min(depthY * 240, 240));  //convert to 320, 240 space

    int colorX, colorY;
    ImageViewArea iv = new ImageViewArea();
    // only ImageResolution.Resolution640x480 is supported at this point
    nui.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(ImageResolution.Resolution640x480, iv, (int)depthX, (int)depthY, (short)0, out colorX, out colorY);

    return new Point(colorX, colorY);
}