Question

I am looking for way to (as quick as possible) get the corresponding depth to a color pixel from the Kinect camera.

I have found the MapDepthFrameToColorFrame function. But that only gives me the color at a certain depth position, I want the opposite.

The reason I want this is that I will be able to click on a position at the RGB image and get the position to that pixel.

Is there a way to do this faster than looping through all results from MapDepthFrameToColorFrame?

Answer 1

The problem here is that not every color pixel will have a depth assigned to it because of the way the cameras and IR emitter are positioned. If you’re facing the Kinect and holding up your left hand in front of your chest, there’s a section of your chest that won’t have depth data because your hand blocks the IR emitter. Since the IR camera is closer to the RGB camera, you generally won’t have the same problem going from depth to RGB; if the Kinect can see the IR, it can see the RGB.

To answer your question: I’m not aware of any way to do this because the RGB index doesn’t give you enough information to calculate the depth index. Have you tried using the MapDepthFrameToColorFrame and looping like you mentioned? Unless you’re clicking like a maniac (or a computer), I’d be willing to bet that the application will be able to handle this just fine. I wouldn’t spend time trying to optimize this until you’re sure it’s causing a problem.

If you want to, you could try cutting down on the size of the loop by checking a subset of the color coordinates returned by the function. Get the coordinates of the RGB pixel (x, y), then check the depth coordinates somewhere close to (x,y). Perhaps (x-20, y-20) to (x+20, y+20). If that fails to find it, you can check the rest of the array. Test this out a bit and you can probably find a good range to check around the requested pixel that you’ll get a hit the majority of the time, without a lot of looping.