Hi Sam, I guess we are not all aiming to "just" have an electronic device able to capture realtime 3D. The biological retina is not working as a structured light camera manner (nor as the ToF either): it is much efficient in several aspect and is based on passive-traditional-stereo-binocular vision. The retina is not outputting "frames" or images and has a much higher temporal bandwidth. These are some few reasons why we should keep in researching almost classic stereo. I have also no doubt there are plenty other reasons.

Kinect is a very well optimized tool, using a sort of "Compressed Sensing" when only a limited series of structured patterns are projected for close object indoor applications.

For different purposes we use different equipments: binocular vision of small or wide basis distances, structured light for close objects, TOF for mid-range objects, LIDAR in a large radius open-air etc.

Active sensing "pollutes" the scene, as it throws some sort of light onto it. While this appears unobtrusive at first (it's just a matter of choosing a wavelength outside the range of the visual information, e.g., infrared for RGB imaging), it's quite hard to solve the problem when more than one device scans the scene. For example, think of a team (perhaps a swarm) of robots dispatched to recover the 3D structure of a building: each structured light source disrupts the scanning of surrounding sensors. Also, sometimes a wavelength that does not interfer with an RGB sensor may actually disrupt some elements of the scene -- say, for example, wildlife imaging, where infrared and/or UV lights fall in the sensing range of several species, or for military use, where light sources of any kind cannot be used. Finally, stereo vision works very well for any distance to the scene, while active sensing depends on a powerful source for, say, imaging a scene several kilometers apart, and is essentially impractical for astronomical scenarios.

I think the answer has already been given in the previous posts, but here are my 2 cents. Have you ever encountered a glass door using any active sensing? Well, this makes the sensor go nuts and you can't really tell their depth accurately, thus the RGB-D advantages are lost. So transparent surfaces are a big no-no for this kind of tech.

Also, our brain is not just getting queues from stereo vision, we also have the depth-from-defocus running at the same time. I'm betting that some kind of estimation is done via sound. The computational model for traditional computer vision 3D sensing is really different from what our brain does.

Neither is better or worse per se, it just depends on what you want to achieve on your project.

My view is that all methods have their pros and cons, and in general you try to use the best tool for the circumstances. Structured light is great for short-range, high-precision full-field 3D imaging, but that is often overkill and inefficient for some operations. Passive binocular vision is good for rapid range estimation e.g. obstacle avoidance, catching moving objects, relative range, size, and speed estimation (close, slow, small objects vs distant, fast, large objects), and making efficient use of the powerful and "free" passive lighting and information it provides. Making use of passive (and usually static) lighting also gives consistent information from world coordinates unlike active lighting that moves with the sensor.

Even for 3D imaging we need to be careful on generalizations. Are you tracking objects or doing surface inspection? Structured light is great for precise 3D modeling, but it produces enormous amounts of data that need to be processed. If you are just tracking an object or getting pose estimation, you only need and want a very small data set (for fast computation). If power efficiency and field-of-view are concerns, a wide-FOV structured light sensor wastes a lot of optical energy. Even a focused scanning laser will generally cover a wider FOV with more precise 3D point measurements in sufficient numbers to optimize precision, power, and computation speed for applications like pose estimation.

Ultimately, I believe it depends on what the 3D data is for and optimizing the trade-offs. For applications with short range, small volume, high precision, full surface, large point count, reasonably controlled conditions, 3D imaging, I absolutely agree that active structured lighting tends to be much better than passive binocular imaging. Of course, using binocular cameras with structured lighting should, in principle, be a little bit better because of the doubled measurement information.

Both "active sensing" and "passive sensing" or even "mixed sensing' are useful for distinct purposes. These techniques mimic some sorts of "human vision", but cannot

achieve the whole range of the power of "human vision",still. This inspires the research

areas for understanding the actual "human vision" and seeking methods for even better "artificial vision".

I'm really pleased and motivated by the great responses to my question and appreciate the thoughtful and detailed analysis. My team at CU-Boulder and University of Alaska Anchorage has been working on the idea of Computational Photometry to build a binocular vision system with I/O path co-processing that is bio-inspired using FGPAs (we just got our custom PCBs back and we are testing now), but we keep challenging ourselves to make sure this is a worthwhile endeavor. At times I've thought that perhaps trying to build vision systems more like the human visual system is like building a flappy wing (ornithopther) compared to fixed wing airfoil (RGB-D) - i.e. less practical, but more interesting. On the other hand, I can see as many of you pointed out that there are significant limitations to active sensing with structured light (range is a big one and the IR interference with with the target being observed). We plan to share our hardware design through an open hardware university program in the future and hope contribute to the general research community with shared interest - I will certainly post updates here in case anyone has interest in our reference design. I also realized that if we wanted, we could integrate an active depth mapper variation of our CP (Computational Photometer) by using a near IR sensitive CCD in place of one of the optical channels and adding an active light projector (e.g. TI DLP) - basically building an FPGA version of PrimeSense (we'd like to get to 1 mm accuracy at 1 meter for close range). If anyone has pointers on papers or research to read regarding the comparison of both methods, that would be much appreciated. Again, thank you all for the great feedback!

This article that appeared in the December 2013 IEEE RAS is why I started to question the value of our binocular CP - http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6678594 - I was of course familiar with the Kinect and RGB-D before, but only recently became familiar with OpenNI and the wider range of solutions out there now in the market place. Most appear to also to have limited ranges of 1 to 5 meters, but accuracy limited to 1% at best (great for gesture recognition), but not so great for texture mapping and more scientific endeavors or the understanding of human perception. Certainly underscores how amazing human perception is given the wide range of operation and accuracy.

I have a personal bias in this matter. Carefully controlled passive sensing hybrid full frame techniques can be as good as active sensing.

The Kinect has become the defacto tool for developing all kinds of processes which exploit the range map information. Why? Because it has a very good SDK, community support and cheap hardware. It has several shortcomings...range, ambient light and resolution are among them. Also, operation through an optical interface is limited as cited in one of the other responses.

A similar inexpensive passive stereo imaging system would accomplish the same thing. It would be a catalyst for development of algorithms to transform binocular imagery to range maps. There are a few shortcomings. They boil down to inability to handle texture properly. Both lack of texture and too much texture are problematic. Every workable tool is processor intensive.

It would seem that if the hardware implementation was limited to primitives leaving any level of complexity to being in the application layer, it would serve both worlds. I am working with a system that uses two cameras and a structured source. I find it more versatile than the single camera-single source architecture of the Kinect. I also find that by placing the source between the cameras, the system improves the coverage which is not occluded. The bottom line is that a good passive system can be used for a structured light system but a good structured light system generally cannot be used passively.

There are a lot of aspects, most already answered. Recently I wrote a short text on this topic with some focus on the costs. Trying to summarize I would say:

- binocular - a camera costs nothing (1$ for VGA camera module with a lens) so 2 are also nothing. Calibration and lack of robustness are the issue.

- structured light - a camera + 1 LED+pattern - cost probably less than 2 cameras. Calibration is easier and it is robust in most situations. Dominating the consumer electronics today (Kinect, Leap Motion,...).

- time of flight - does not require baseline compared to the previous two. I would bet on this one from the technology point of view, however it is more expensive and the question is will it ever become cheap enough (I mean 1$ range).

The new Kinect on XBox One is based on time of flight instead of structured light like the original Kinect. This may commoditize and cheapen time of flight technology, much the way the original connect commoditized structured light solutions.

Thanks for the updates on the new Kinect. I have ordered an ASUS Xtion v 1.09 for research purposes to compare off-the-shelf to our custom designs. The Ocipital mobile active RGB-D also appears interesting. My goal is to compare all off-the-shelf options to what could be built with open hardware through university and open programs. We intend to make our design open and to share it in late 2014. We'll be presenting more details at SPIE Technology + Applications in May this year. It still seems to me that it comes down to range and accuracy required at a range along with how robust the depth map is from both methods, but I certainly see the value of passive 3D and plan to stay the course on this - our system can be combined with an add on TI DLP projector for anyone wanting to also look at structured light methods for RGB-D. We believe that the computational interface - essentially a CVPU (Computer Vision Processing Unit), much like a GPU is a key element. It seems that industry agrees based on recent OpenVX standard, but not sure how open it really will be compared to OpenCV, OpenNI, and PCL for example and current software solutions for CV systems, so we wanted to focus on and effort for open hardware and firmware to help enable research, education and innovation. Please join our effort when we release our design if you agree.

You may also want to include Softkinetic, which uses time of flight. Because of this, it has a different noise profile and accuracy behavior than the structured light solutions.

Thanks - I will look into that. Overall it seems that methods for recovering 3D information that could be considered, include:

1) RGB-D active (structured light projection and observation)

2) Binocular passive

3) Structure from motion of a single channel (e.g. from a UAV at altitude)

4) Time of flight methods

There are clearly more depth cues even than this used in human vision systems for passive perception.

My interests include both machine vision (cheating as I see it compared to the human analog) as well as computer vision (more faithfully emulating the human system or biological at least).

I really appreciate all the interest in my original question and have learned quite a bit through this dialog. Hopefully of value to the group as well.