In real neurons, energy consumption is mainly determined by the efflux of potassium from the cell and sodium inflow in the cell that needs to be pumped back, with a small addition of chlorine and calcium flowing in during synaptic transmission. During an action potential, this is fairly linear with the total membrane depolarization. See e.g. Howarth and Attwell 2012 http://www.ncbi.nlm.nih.gov/pubmed/22434069 . As a rough general measure for cost, you could maybe consider depolarization of the membrane?

Thank you Bas-Jan, you are talking about energetic requirements for cellular processes. By "computational cost", I meant the number of variables

needed to be stored and the floating-point operations needed for one-step update.

I would agree with Joachim that Big-O notation is the sensible way of making comparisons. You should probably consider whether you want to compare the computaitonal costs of training schemes or of execution. I am going to assume it's execution, let me know if it's otherwise. You need to identify the fundamental operation common across the diferent neuron models (executed repeatdly) and then  find the position on the Big-O scale. Starting with the seminal backprop algorithm I would suggest weight * input multiplication. You will need to look at theother neuronal models you are itnerested in to see if this fundamental operation is common to them.

Eric,

Be mindful that for example the Hodgkin and Huxley's models the flow of current through the nerve fiber. To determine the computational costs would be in my opinion(and I may be wrong but I do not think it has a canonical implementation such as for example:backpropagation) totally dependent on the implementation of the model and not the model itself.

It would be an interesting question to ponder if you could model the complexity of the system and translate those to energy consumption estimates to see if you could find a correlation between simulation and real energy consumption in neurons(as per Bas-Jan Zandt post).

Arturo, you may be right.

My original question was motivated by this article (link below):

http://enu.kz/repository/2010/AIAA-2010-3540.pdf

I recently developed a neuron model and I would like to calculate its computational cost to show that my model is more efficient (from a computationally point of view) than the others, such as the Hodgkin & Huxley, or the one proposed by Izhikevich.

Please check page 12, section V (Computational Cost) and you will have an idea what I attempt to do. Energy comsumption sounds interesting, but I'm currently focused on computational efficience.

Best regards!

Erick

Erick,

Thanks for the reference, it puts the conversation into context and provides a clear framing for your question.

To determine whether your solution is optimal you need to place all models on equal footing. As with the paper you have referenced they benchmarked it using different differential equation solvers. You must first determine if your model can be put on equal footing with your baselines.That is can they be solved using the same time steps or could you have a stiffness problem. This may imply an efficiency vs accuracy.

Hope this helps

I think that is part of the problem. 

My neuron model does not use differential equations. If you don't mind, I would like you to check my article "The Neuroid. a novel and simplified neuron model". You can download directly from this page.

I tried to contact Professor Long to get any help in this regard, but to date, I have not got an answer.

I will appreciate all your support in this regard

I will try to answer this as best as possible and please note that this is my opinion and does not necessarily reflect consensus.

I will take as initial point the paper by Abbott et al.[1] where he positions  Hodgkin and Huxley's(H&H) model and  Hopfield along with McCulloch-Pitts on the other end. These extremes in my opinion are non comparable because at the one extreme is biological plausibility and at the other computational efficiency or other considerations. For example you cannot compare the delta E in [2] with the voltage evolution of H&H algorithmically since they will require a completely different algorithms and they may vary considerably based on the authors selection of parameters.

If you look at the work of Long et al. [3] he started with all models at the same representational level to carry out the comparison. What I would suggest is to restate you model in terms of the voltage time evolution and do the complete path to simplification and then carry the H&H through a similar path until you can compare them at the level you desire.

Hope this helps.

[1]Abbott, L. F., & Kepler, T. B. (1990). Model neurons: From hodgkin-huxley to hopfield. In Statistical mechanics of neural networks (pp. 5-18). Springer Berlin Heidelberg.

[2] Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8), 2554-2558.

[3]Long, L. N., & Fang, G. (2010, April). A review of biologically plausible neuron models for spiking neural networks. In AIAA InfoTech@ Aerospace Conference, Atlanta, GA.

Eric,

First of all, you didn't provide a link to your Neuroid model. It is interesting to take a look on it. If you can, please share your paper also.

Second, "computational cost" is related with realization. You cannot say about efficacy of the model from equations or any logical representation. It is possible to design an extremely fast realization of complicated model (like H-H) and extremely slow version of simple model (like LIaF).

Please take into account that for modern CPUs, operator 'if' may be costly than exp function! For example, it is true for C/C++ in both MS Studio and GNU GCC, but I'm not sure about Python or Matlab.

Dear Ruben,

You can download the article of the Neuroid from this page (just in case, I attached the article to this message).

I just found out that there are several methods to approximate to differential equations, such as the Forward Euler method and the Runge Kutta 4 method, but I'm not sure if operators like "If" are more expensive than differential or exponential functions...

I'm trying to implement several neuron models (including mine) in MatLab by using the Forward Euler method to measure the elapsed time  of each algorithm. Same time span (500 ms) and same resolution (delta t = 1 ms).

What do you think?

Dear Erick,

Thank you for the paper.

Let me explain in simple example. CPU has, so called, registers. If you write 'if' statement in your C/C++ code, like this:

if (vol > 30) {

do-something

}

compiler, usually, saves registers in a memory(puts them on the stack), i.e. accumulator, and all memory offset registers and so on, so on... are sent into stack and only after that do-something. In fact, CPUs needs several cycles (about 7) to just go into 'if' body. On the other hand float-point co-CPU calculates exp function for 2 processor cycles..... To avoid this paradocsal problem, C and C++ has special operator:

a = (vol>30) ? some-computation : another computation is volt less than 30;

which doesn't require stack operations. Many other languages have similar problem, I think. Because Python also has two different notations for 'if'. One is like this

if vol >30:

     do-something

and an other like this:

a = some-computation if vol > 30 else another computation is volt less than 30

For Matlab/Python/Java and all other interpreters situation are different. For example, you cannot say how Matlab does computations, and you cannot estimate efficacy. 

Related to Forward Euler, RK4 and other numerical methods, be sure that you use classical approach: tabulating of right side of differential equations. In this case every step of Euler is just one subtraction, rounding and memory reading. Don't forget that for first order ODE Exponential Euler is better than just Euler. The accuracy of exp-euler's solution is about RK2, but with right-side tabulation, it needs few processor cycles, about the same as one 'if' operator.

Hope, it helps.

Dear Ruben,

Thank you for your answer. Please take all the time you need for reading my article.

Basing on your answer, do I need to translate the Neuroid to C or C++ in order to get a good estimation of the computational cost of my neuron model??

I read this article a couple of weeks ago (please see the attachment) and I would like to test my model based on that methodology. Unfortunately, It's been a long time since I programmed in C++

Any suggestion?

Oh, Erick,

I'll try to read your paper and reply soon.

To answer your question, I would say: if you would like to show an efficacy of your model, you have to make several realization under different platforms. If it is a neuron model, think to check under: C/C++, Python (pure and with libraries), Matplab, XPP, and NEURON. If you are thinking about hardware realization, please add VHDL or VerilogDL and some simulator of FPGA. Please don't do the 'standard mistake', Matlab isn't only one. isn't the best, isn't most effective and isn't a good choice. Don't limit your-self.

In addition to computational efficacy of neuron model, please think to make good battery of tests. Which known phenomena of neural dynamics your model can reproduce?

Ok Ruben, I will take into account your suggestion.

My neuron-model is in line with the one proposed by McCulloch and Pitts, but unlike the latter, it relies on the assumption that deterministic Pulse Frecuency Modulation (PFM) resembles the impulse generation process of those neurons which fire at an almost constant frequency (tonic or pacemaker neurons). Therefore, this model is able to represent essentially tonic-spiking behavior.

I made this oversimplification in an effort to provide a first approach to neural dynamics for my undergraduate students and I would like to test its computational capabilities. Last year I built a Neuroid-based retina model and it yielded interesting results.

Please let me know your opinion about the article

Best regards!