I want to detect an moving object using Kinect’s IR sensor and overlay the depth profile view onto the webcam view from the kinect. Ideally, I would also like to register the kinect pictures with an external webcam.
Any examples on how to get started?
If you want to match the RGB and depth streams you need to do stereo calibration. Regarding the Kinect itself, it depends on what library/driver you’re using as the functionality might be already built for you, allowing you for get the proper RGB value for the same x,y value on the depth image.
For example:
Also there are wrapper libraries that simplify things a bit. For example ofxKinect as a setRegistration() method.
If you want to use an external webcam, you would need to calibrate it first. You can find some handy articles on Niccolas Burrus’ wiki. I also recommend having a look at the RGBDToolkit project since it does what you’re looking for, but with a DSLR camera. The Principle is the same though.
Here are a few more resources on calibration: