Nowadays, the multi-agent systems theory begins to be quite developed to become a serious candidate for the model of visual system. The components of heterogeneous agent type, parallel or concurrent processes having different roles in vision, in millions of copies, can model the visual system. The cooperating and communicant agents can run individually functions by assuming identical or different roles in the agency, hierarchical or not hierarchical. An agency of multi-agent sistems may be also accepted as a model for the visual system.

Off topic...

114 Views, 6 followers, but very few have clicked the green arrows.

Don't forget to "like" interesting contributions, including the original question :)

@Peter

Any visual scene can be described more or less well through text (not necessarily visual literal symbols). If we could decode the neural field representing a visual scene (an image or a series of images) automated by computer using functions or coupled functions into a descriptive text composed of sentences, we already build at least a part of a visual system. The obtained propositions can be the entry of a decision-making system for a human prosthesis or an artificial vision system.

You ask me if I have these functions or algorithms? I do not, but someone could create them, if he would well funded for this purpose.

Nature does not create complex patterns, but ingenious. They appear simple after we know better. Only combinations of simple models give the apparent complexity (fractals, for example). The DNA now seem something extremely complex than 30 years ago? Sorry for the less technical opinions.

To go back to the original question, it is not Kurzweill that determined that the cortex is based on a column architecture, but Sir John Eccles back in the "80's". I question the estimate of how many neurons are involved, as it seems somewhat light. The actual number of neurons has been somewhat of a debate, since primate brains have smaller neurons than for an example mouse brains. it is only recently that there has been an advance in estimation of brain size in neurons, the ability to create brain soup, has advanced our understanding of the complexity of the brain significantly. It has not yet been used to do comparisons between different areas of the brain.

Deep Learning studies have suggested that we can estimate the complexity of the brain using vectors as suggested in this question, but to do so we need the brain to be homogenous with respect to function. Theoretically according to deep learning, the brain uses the same mechanisms across a wide range of areas, which simplifies the software enough to allow for easier design. It is not just how many functions that is important or what the functions are, but the throughput in time, it is hard to implement a sufficiently fast algorythm on a single gpu. Remember the brain is massively parallel in structure, and gpus are serial.

My question comes from the reading Poggio and others works, in particular: " A Theory of Object Recognition : Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex " in : T. Serre , M. KOUH , C . Cadieu , U. Knoblich , G. Kreiman , T. Poggio

In this work, it seems that, in order to recognize an object, it is "sufficient" to go through 4 layers of pairs of operations: dot product + sigmoid for simple cells, and a " softmax " for the complex cell. This to allow to the neurons of the IT areas to be able to recognize individual objects .

The strange thing (but I may be wrong here!) is that by applying this model to the theory of cortical minicolumns would seem to be sufficient onlya pair of "vector-scalar functions" for each minicolumn of the visual cortex from areas V1 to IT!

This would imply a reduction, in the number of operations required, by 700-800 million (estimated number of neurons that make up these areas) to only 10-20 milion (some operation multiplied the number of minicolumn involved) ....

Obviously I agree that is not Kurzweill the father of the theory of cotex organization in minicolumns, but I mention him because in his recent book : "How to Create a Mind: The Secret of Human Thought Revealed”, he supports the hypothesis that the entire cortex is organized modules consist of mini-columns devoted to recognize shapes . He speculates that in our brain there are about 300 million of these modules.

So in essence what I am asking is: it may make sense to think that the cortex is organized in modules of about 80-100 neurons ( minicolumns ) whose entire function can be approximated with only a few vector-scalar function ?

Ah, mini-columns, that is an even later convention. I believe it was LaBerge that discovered them. You have to take the number of neurons per mini-column with a grain of salt, if only because the early staining techniques didn't stain the smaller interneurons, but I agree that the mini-column is probably the basis of processing in the cortex.

Part of the issue that the brain has to deal with that a computer doesn't is uncertainty in the mapping of the locations, which means that the mini-column mostly allows for the uncertainty and creates a composite "Function" that resembles a shape sensitivity. This is indeed the realm of deep learning.

I must caution you again not to assume that because you can do the calculations for the algorithm in a single gpu that you will be able to get away with that minimal an implementation, each neuron works in parallel, and while the mini-column seems to work as a composite, each mini-column also works in parallel, so you will need to have probably more than one gpu to process the serial calculations in time to be useful. In fact current wisdom is that you need a rather large array of gpu's to get the speed up to a useful level.

Each layer of the visual areas seems to be dedicated to detecting different shapes and abstractions. For example the bottom layers are sensitive to dot and lines of various orientations on the way up, so although there are columns there must be significant amounts of cross talk between columns. These ideas are based on some original work done in the 60's by:

Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of physiology, 160, 106-154.

They looked into cats brains and found the basics on the structures of the first few layers. This might mean that multiple vector-scalar function might be needed to be used to approximate that column to allow for the cross talk. Also i believe there is some signal recursion.

Taking this a different way the whole visual system might be able to be simplified by using a few well connected columns as a fovea and scanning around the image as computers can tick faster than neurons. This as been done to some extent by Lowe when looking for SIFT features.

Lowe, D. G. (2004). Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2), 91-110. doi: 10.1023/B:VISI.0000029664.99615.94

The system he uses was inspired by the visual cortex.

So taking this into account... i would estimate more than the number of columns so about 8 million+ maybe perhaps times 3. In the hope that approximating this as a bottom, middle and top layer which can cross-link will allow a good proportion of cross talk and signal delay integration. But if the fovea could be reduced and so process a smaller image patch or a lower resolution this figure could be reduced.

I would love to discuss this subject with people as it interests me greatly. So feel free to e-mail.

I hope this helps.

Robert

What the Deep Learning people have been doing, is working with toy versions of the algorithm getting them up to speed, and seeing what use they can get out of them rather than trying to calculate the whole array at once. This way they can approximate the capabilities well enough to check out the theory without going to the expense of trying to get real-time data from a full implementation.

Devis, you might be interested in an association called BICA (Biologically Inspired Cognitive Architectures) where a lot of this work has been reported in their journal, I am sorry that the journal is mostly subscription because it is operated by Elsevier but if you have access to a University Library you can access it via science direct.

You might also be interested in the course by professor NG in Machine Learning. Its available for free from Coursera. One of the topics he covers is deep learning.

The problem is to believe that just by implementing, say, 800 millions of functions, you can implement the behavior of the VI visual cortex. This is an excesively optimistic position of humans believing that our biological system is reproducible by simple machinery. Better to talk in terms of partial simulation.

Like I suggested earlier, the theory requires that the structure of the brain is homogeneous with respect to mini-columns. The successes with Deep Learning suggest that to some extent this is true, but it is not the whole answer. Evolved systems are not homogeneous they have much fine detail that seems random at first glance, and this uncertainty may to some extent add to the processing capacity of the brain in ways that deep learning and other homogeneous systems can't.

"How many functions (vector-scalar) must be implemented to realize a visual system similar to that of a primate?"

I think this question leads us in the wrong direction. It is not the number of functions that will determine similarity to primates but, rather, the structure and dynamics of the minimal neuronal mechanisms competent to perform primate-like perception. For more about this see "Sparse Coding of Faces in a Neuronal Model ..." On my R G page.

I agree with Arnold that the question is misdirected. The more useful question is, what is the specific fuction we are talking and how it is performed by the primate. Also, I strongly feel that the current deep learning approaches are too primitive and do not lead us to any fundamental learning about the visual pattern recognition. In the visual pathway, even at the level of LGN, the feedback connections far outweigh the feedforward connections. I call these active feedback networks, whereas the so called "deep learning" neural networks do not have any feedback connections. Even though we use feedback of error in training the network weights, once the training is over, there is no feedback when it is being used to classify test data.

So, let us ask the question, "How far are we in understanding or closely modeling the primate visual system ?"

All of the answers have addressed the problem of matching the bottom-up capabilities of a primate visual system. But "to realize a visual system similar to that of a primate" is a much broader question. A simple example of top-down image analysis that has already been well implemented is 'active contours' or 'snakes', which hypothesize that there is a smooth curve somewhere in the image, perhaps with substantial gaps, and numerically create a model which slithers around until it fits the visible parts. This kind of seeing things despite incomplete evidence is a task that our visual system is particularly good at (including false positives, like the shapes we see in star patterns), and it must have a much richer system for hypothesis creation and testing than current computer vision does. Such mechanisms are much less local than detection of edges, textures, edges between textures, etc., from column processing, though they must use column output as their raw data.

The answer to "how many?", therefore, is either (a) whatever number these deeper processes need, or (b) no number of functions (vector-scalar) is sufficient, on their own.

Active contours for detection of even incomplete boundaries is a great example. It is in such and similar processing based on top-down inputs that the feedback connections make a huge difference. By their very nature, these prior world knowledge based top-down mechanisms need to involve active feedback to modify the nature of bottom-up processing.

Another good example of such active feedback being needed is colour constancy: we perceive that the colour of an object that is partly in shadow (which reduces R, G and B, and often changes their ratios) does not change at the shadow edge, on the basis of complex scene-modelling cues. The bottom-up processes, on their own, would give a colour boundary. It would be great to know how far down into them the top-down adjustment reaches.

A. G. Ramakrishnan: "By their very nature, these prior world knowledge based top-down mechanisms need to involve active feedback to modify the nature of bottom-up processing."

An example of such neuronal mechanisms is given in "Self-Directed Learning in a Complex Environment" on my RG page. See, in particular, *Searching for Objects*.

Tim : "The bottom-up processes, on their own would give a colour boundary. It would be great to know how far down into them the top-down adjustment reaches"

I believe the answer to that is, that there is no level at which there isn't some feedback!

Graeme: all the way down to the rods and the cones?

I'm not saying this is false, just wondering how strong is the evidence.

To understand the the nature of vision its essential to first study it….as the phenomenon occurs to us. We need to experientially and intuitively get to grips with both the data-sets and the processes of information exchange taking place within phenomenal field. The ontology must be correct? Vision is prior-to science. It occurs to us and not our instrumentation. We have developed a new form of illusionary space called Vision-Space ( as opposed to picture space) from which its clear that vision is almost entirely non-photographically rendered. One where for example binocular advantage to the saliency of vision has nothing to do with binocular fusion - there aren't any 'pictures' to fuse. Critically we discover that the base structure to the phenomenon is actually a 'field' and that this is likely to for the basis for our underlying perceptual structure (something that we as a biological system generate) and multi-sence integration. So what we need to understand is how do we generate perpetual structure and then how is it populated by the various sensory inputs. This will tell us, or at least strongly suggest, what the various neural structures are doing and then how they segment that data from the light array?

We can create moving image Vision-Space media which models visual awareness and are looking get that working at real-time speed where full evaluation can begin. A list of presentations is attached.

What we see is what we guess.

Vision-Space sounds like an interesting guessing scheme.

Going back to the original question Devis, The 'field' structure of Vision-Space is generated from vectors. We establish a radial field around fixation (locus) that provides us with 'in time' spatial awareness via 'proximity cues' (not depth cues). We argue that this capability is 'implicit' deriving direct from the light array and that it's pre-conceptual. Through the 'where' pathway we don't establish spatial awareness through building or compiling perspective and occlusion cues etc (way too slow). The Vision-Space presentations give you all the details.

How often do you rely on that as you operate in the world?

I know I'm always guessing as I operate in the world, even when I think I know.

Its really important that we get inside the guessing game. It's the only game in town! Its not guessing, it actually how we go about forming our relationship with the real. Its the nature of reality? The rest is just building on assumption.

the issue is which and not how many. One could obtain a good response using evolutionary computation. Just simulate agents with certain constraints in a certain environment and see what do they develop. The closer is the environment to ours, the better will be the answer. Let us know the result.

John, we form our relationship with the real by guessing, and then checking whether the guess's prediction come out reasonably well.

It's called perception.

It's called science.

It's called successful marriage.

All, I have just posted this up as a response for 2 other questions so sorry if you have encountered it before, before:

Here is the latest Vision-Space (VS) presentation

Vision-Space: Exploring Implicit Spatial Awareness

http://youtu.be/WMh9j60ERxU

where I try to give some idea of how VS might work with simulation. We have previously rigged the VS post production tool up to a basic depth-map camera and that worked OK, so the the move to real-time VS even at this early stage of development is possible. We are working with Cardiff Uni (computer science) on this.

The presentational is however based around paintings and the ongoing intuitive exploration of phenomenal field. I think that by understanding phenomenal field and perceptual structure we can think again about how we approach and think about (in scientific terms) the visual system and what its doing. If we have something wrong at the level of 'approach' then outcomes from ill conceived experimentation will be confusing, overly complex or even misleading. I have just read Jeff Hawkins' book On Intelligence. While I am not aligned with the 'brain theory' the process he describes for neural processing is likely to be ubiquitous?

For those of you who have watched some of the other VS presentations you can skip the first 3mins or so background stuff or I will just be engendering the 'yawn' reflex!