I want to use RGB and depth video generated from Kinect (not version v2) and extract real coordinates so to map them on point cloud. I do not possess Kinect device but only the data. I am primarily relying upon OpenCV for this but can use some other open source tool. I have extracted frames from both videos and frames from both videos are numpy nd arrays 640X480X3. Can anyone provide some pointers or suggestions or existing solutions please? Thanks a ton!
Hello Saurabh, there are many ways to extract coordinates from Point Cloud. I recommend this link: https://answers.ros.org/question/192489/how-to-extract-the-world-co-ordinates-from-a-pcl/
Or try using the Point Cloud Library: http://pointclouds.org/documentation/
Hope this helps!
If I understand correctly, you have an RGB video and a IR depth video generated and saved from a Kinect, right?
the RGB video is only necessary if you are going to be colourizing the depth cloud.
As for openCV... I'm not sure why you will be using this. openCV is very useful for modifying images (contrast, threshold, etc) to get contours and such. I would ignore that until you know how to get the depth information from a still image.
You may want to visit this page, and skip down to where Shiffman discusses pointclouds: http://shiffman.net/p5/kinect/
He discusses how to build a depth lookup table (depends on which version of the Kinect your data is from) and then generate real-world coordinates from that.
The reason I suggest that part of the page is because it doesn't access the Kinect for anything.
This will, of course, depend on how the depth video was saved, whether it is just grayscale or if it has the actual depth information in it (ie: the image 'looks' like a grayscale, but the values of pixels go beyond 255)
good luck.
@Thadeu Brito: actually my question was extracting real (world) coordinates from RGB and depth videos saved from Kinect v1 and NOT from the cloud. Apologies, if my question appeared other way. I have extracted time-aligned frames from both videos and RGB frame has color info whereas depth frame, saved as RGB, has depth information encoded in 11 bits, 8 bits in B channel and 3 bits in G channel. How to retrieve world coordinates using this and the camera intrinsics and extrinsics for a human face? Thx
@ Gerry Straathof: Yes, you are absolutely right! RGB and depth videos saved from Kinect v1. I have extracted time-aligned frames from both of them. RGB frame has color info whereas depth frame, saved as RGB image, has depth information encoded in 11 bits, 8 bits in B channel and 3 bits in G channel, R channel has value zero throughout. So the depth gives me world coordinate Zw after applying camera intrinsics and extrinsics. How to retrieve world coordinates Xw and Yw using this and the RGB camera intrinsics and extrinsics for a human face facing the Kinect? Thx a lot for pointing out Shiffman page; will look into it now. Hope you could give some more pointers or suggestions.
@Saurabh I am also working on kinect dataset. Can you please help me to get colour depth frames from RGB videos which are xef format.
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation