Is there any rule of thumb how average activity of a network scales with its size? For example, a linear relation would mean that with increasing number of neurons the average number of spiking neurons also increases.
I don't think so, the activity depends on the connections. If you train a two layers feed-forward nn in a supervised way for classification the activity will not scale with the number of neurons. If you look instead to some random connection NN or something like liquid state machine maybe you can find some relation between activation and number of neurons.
Well first it depends on the task and I would call it on the information load. For instance looking into SOM like networks, one can determine the maximum storage capacity. This means that increasing the network size will not increase its activity. However, for a given task you should be able to determine a minimum size - similar to the SOM example. Second, it also depends on your task. Various networks types shows different dependencies. A highly non-linear problem will require more neurons than a simple linear extrapolation. So far I have not seen such approximation in current stat of art research. Could be interesting to study tho.
thank you for your answers. I was more referring to the relationship between network size and activity in natural neural networks, but your answers clearly pointed to the fact that the task performed by the network might be independent from the network size. That is, with increasing network size the task may or may not change which would or would not increase the activity. Developmental biology mostly measures the brain/body-ratio which is, I think, the highest for humans, although there are larger brains (in absolute size) than those of humans. This indicates that task and network size have to be treated as separate variables.
Yes indeed, I would even go further and say that the size is only the result of required structure. But this is also not so straight forward, because as you said there are various animals with brains larger than humans but with fewer cognitive capacity.
- Badges
- Science topic