The past few weeks I've been analyzing the particular solutions generated by my ANN. I've seen that even for simple tasks like a delayed XOR function the solutions can be highly convoluted and highly distributed across many different neurons making it really difficult for the human eye discern what specific computations are happening across this network. (See the weight matrix below which solves a delayed XOR task. The solution happens to be beautiful but is very complex and highly distributed for this simple function.)
These observations signal to me just how daunting the task of reverse engineering real neuronal networks are where:
- I can't control every aspect of my experiment
- There are stochastic elements
- Complexity orders of magnitude beyond simple ANN's
- The specific computational function isn't known
- The physiological details aren't fully known
- etc.
Hence, it is clear to me that work needs to be done in developing a theory for making sense of distributed computation in dynamical networks. Has there been work on this? Am I the first one to ask this question (or is this Dunning-Kruger in motion)
If you would like a non-standard, somewhat heterodox, take on connectionist (ANN) computation, I would recommend looking up analogue computational theory that includes "structural representation". It's 'analogue' not in the sense you would assume.
https://www.academia.edu/228839/How_do_Connectionist_Networks_Compute
Dear Akiva! Please read the article in the Attachment.
Vladimir
Dear Vladimir, very good article by Gong and van Leeuwen!
Maybe the following papers will also help, dear Akiva:
Buesing L, Bill J, Nessler B, Maass W. Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons. PLoS Comput Biol. 2011;7(11):e1002211. http://journals.plos.org/ploscompbiol/article/asset?id=10.1371%2Fjournal.pcbi.1002211.PDF
Moreno-Diaz A, de Blasio G, Moreno-Diaz R Jr. Distributed, layered and reliable computing nets to represent neuronal receptive fields. Math Biosci Eng. 2014;11(2):343-61. https://aimsciences.org/journals/pdfs.jsp?paperID=9180&mode=full
Kunkel S, Schmidt M, Eppler JM, Plesser HE, Masumoto G, Igarashi J, Ishii S, Fukai T, Morrison A, Diesmann M, Helias M. Spiking network simulation code for petascale computers. Front Neuroinform. 2014;8:78. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4193238/pdf/fninf-08-00078.pdf
Hahne J, Helias M, Kunkel S, Igarashi J, Bolten M, Frommer A, Diesmann M. A unified framework for spiking and gap-junction interactions in distributed neuronal network simulations. Front Neuroinform. 2015;9:22. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4563270/pdf/fninf-09-00022.pdf
Have a good day, Martin
If you would like a non-standard, somewhat heterodox, take on connectionist (ANN) computation, I would recommend looking up analogue computational theory that includes "structural representation". It's 'analogue' not in the sense you would assume.
https://www.academia.edu/228839/How_do_Connectionist_Networks_Compute
- Badges
- Science method