FPGA is great thing. In many application like robotics and video surveillance we need real time. But modern processors and GPU cant do good algorithm in normal resolution in real time. In robotics there is energy limitation and GPU often too exhaustive.

I think now developers needs fast primitives in FPGA.

Our team agrees - we are interested in overall power efficiency for an image transform and the latency for real-time computer vision. The other issue we see with the GPU acceleration is the latency and inefficiency of DMA from the camera to host memory, back to the GPU for vector processing, then back to host memory again - we'd like to perform transformations like Hough and stereo registration methods for example in the camera data path.

We do plan to test energy efficiency and performance and for cloud based video analytics, the GPU co-processor might be well suited for scalable solutions (but should be evaluated), so the application of Computational Photometer is focused on mobile, battery powered computer vision. On the scalable side we're looking at and comparing many-core, GP-GPU, and FPGA co-processor efficiency.

I also believe that there is space for a Computer Vision Processing unit that is very close to the camera (so it has low latency, high parallelism and low power consumption).

Specially for mobile devices, the transfer of data to other computing units (e.g. the mobile GPU) is a serious bottleneck in terms of latency and sometimes bandwidth.

If the most expensive pixelwise computations (feature extraction, filtering) could be done by an specific processor, that would be a big advantage. However, I would like to see a CVPU flexible enough (and programmable) to include the new algorithms/operators proposed by the researchers and developers. If a lack of flexibility is present, we are back to the current Image Signal Processors that are able to do certain tasks (demosaicing, denoising, even pixel binning) but they don't really help in complex CV applications.

Summarizing, I think it is a very nice project.

Thanks - I will make sure to post a pointer to the open hardware design files and hope to make options too so researchers and educators can just buy a board much like Raspberry Pi at a reasonable cost (similar goal to put an income back into CV research as a non-profit). You should be able to plug our adapter board into and development and education FPGA board with GPIO that can handle the rates and we will provide starter/example Verilog code. My goal is to make sure that this can be replicated globally by researchers and educators that have a passion for CV with hardware acceleration and so they can implement and test their specific transforms and algorithms. I'm really curious if there are neuromorphic transforms and hybrid FPGA/CPU algorithms that could be developed on this type of vision system more easily. What do you think?

Some people would say just use this -http://en.wikipedia.org/wiki/HTC_Evo_3D or this - http://everio.jvc.com/gs-td1/, but we wanted something that is open and accessible to researchers and educators. Clearly though the hardware is out there and coming down in cost, so our goal is to enable the software development and FPGA acceleration of key low-level transformation.

The short answer is yes, an open hardware binocular camera with an FPGA coprocessor would be of use. You should consider making the hardware primitives compatible with OpenVX, a proposed API for computer vision that is compatible with OpenCV.

The OMAP processors usually have a DSP and many have GPUs. The ARM cores each have a NEON which is an SIMD that can accelerate integer arithmetic. Building FPGA primitives that best exploit these hardware accelerators on board the OMAP would greatly enhance the performance.

If you are going to use the OMAP4 or OMAP5, both contain a goodly number of hardware implemented primitives in the Ducati core. The Ducati SDK will require a NDA with TI. Some of this can be exploited by extending the OpenMAX IL to OpenMAX AL.