Question

I’m trying to implement a Real-Time Strategy control scheme for the MS Kinect.
So far, I’ve got a cursor, which can be moved by moving your left Hand (or right, dependant on your handedness). I’ve got an Open-NI-based Kinect controller which sets up a skeleton for player-movements and delivers the wrist-, elbow-, shoulder- and body-center-coordinates to my application.
To project these wrist-coordinates to the screen, I’ve set up a Rectangle, which is situated slightly left/right from the player’s center and as long as the wrist moves inside the rectangle, the cursor moves on screen.
My problem is, that the XNA-Rectangle has the upper left corner as point of origin, i.e. the X-axis points right, as it “should”, but the Y-axis points down, while the Y-axis of the Kinect – coordinate system points up. This results in the cursor moving upwards on screen, when I move my hand down and vice versa. There’s no way for me to change anything with the Kinect-coordinate system, so is it possible to ‘flip’ the ‘coordinate system’ of the rectangle, so that it’s Y-axis points up,too?

Here’s the relevant code:

(from Calibrate()-Method:)

List<Vector3> joints = UDPlistener.getInstance().ParseCalibCoordinates(data);
//0 = Right Wrist 1 = Right Elbow 2 = Right Shoulder
//3 = Left Wrist  4 = Left Elbow 5 = Left Shoulder
//6 = Center
height = 762; 
width = 1024;            
switch (hand)
{
    case 0:
    cursorSpace = new Rectangle((int)(joints[6].X * 2) - 200, (int)(joints[6].Y * 2) + height, width, height);
    break;
    case 3:
    cursorSpace = new Rectangle((int)(joints[6].X * 2) - 1200, (int)(joints[6].Y * 2) + height, width, height);
    break;
}

public Point Cursor(String data)
{   
    List<Vector3> joints = UDPlistener.getInstance().ParsePlayCoordinates(data);
    //0 = Right Wrist 1 = Left Wrist 2 = Center
    double mhx = 0; //main hand x-coordinate
    double mhy = 0; // main hand y-coordinate
    switch (hand)
    {
        case 0:
        mhx = joints[hand].X;
        mhy = joints[hand].Y;
        break; 

        case 3:
        mhx = joints[hand-2].X;
        mhy = joints[hand-2].Y;
        break;                    
    }

int x; 
int y;
if (Math.Abs(mhx - mhxOld) < 1.0 || Math.Abs(mhy - mhyOld) < 1.0) 
//To remove jittering of the cursor
{
    x = (int) mhxOld * 2;
    y = (int) mhyOld * 2;  
}
else
{
    x = (int) mhx * 2;
    mhxOld = mhx;
    y = (int) mhy * 2;
    mhyOld = mhy;
}

Point cursor = new Point(0,0);

if (cursorSpace.Contains(x,y)) 
{
    cursor = new Point(x - cursorSpace.X, y - CursorSpace.Y);                    
    lastCursorPos = cursor;
    return cursor;                    
}

Sorry for the wall of text, I hope, I could make myself clear.

Thanks in advance,

KK

Answer 1

I use an extension method for converting OpenNI coordinates. The following example maps the OpenNI coordinates to XNA coordinates in a 640×480 rectangle in the top left corner, represented as a Vector2 object.

public static Vector2 ToXnaCoordinates(this Point3D point)
{
    return new Vector2(
        point.X + 320,
        (point.Y - 240) * -1);
}

The magic that flips the y coordinate is the * -1 part.

If you want to reach a rectangle of different size than 640×480, you need to scale the coordinates accordingly after conversion. Example:

public static Vector2 ToScaledXnaCoordinates(this Point3D point, int rectSizeX, int rectSizeY)
{
    return new Vector2(
        (point.X + 320) * rectSizeX / 640,
        (point.Y - 240) * -rectSizeY / 480);
}