Many methods were implemented to measure the performance of neural networks such as MSE , NMSE , RMSE, R square for regression. And TP rate ,FP rate , F-measure , accuracy , precision and recall for classification. So on your opinion , which of them are the best. Please explain with details. Thank you in advance.
Regards
My answer is not supposed to be impolite.
But IMHO: Your question does not make that much sense.
What measure is most useful to evaluate a predictive model should be dictated by the specific application it is used for. So think about your problem, talk to domain experts, then decide what makes most sense. NNs actually do not even enter that thought process yet.
Now a different question is what loss you optimize internally in the model. First of all, I try to distinguish in terminology here. I try to talk about "outer losses" (dictated by the application, not the model, see above) and "inner losses" (optimized during model fit).
Usually it is good to try to match these 2 guys, but we all know where this is hard / not so good. because e.g. the outer loss is numerically hard(er) to handle. ML then teaches us a lot about what efficient approximations could be, that we can handle better.
HTH
Bernd
Hi Jabar
The performance of any machine learning model is an active area of research.
There are various performance measures available in the literature.
Apart from ROC, error rates are an important metrics.
But for a specific applications, look at how your neural network model fits into the problem. Look at the P value of the model. That will precisely tell you how the ANN model fits your problem.
Remember, there are no rules set on stone.
So do more experiments and explore more.
Thank you.
I hope this helps you.
Dr. Indrajit Mandal
Thank you prof . Bernd ,
I really appreciate your valuable time and efforts that you spent. I asked this question to share your ideas and explanation with my students. I gave them this question as assignment. So we will explore all the answers in a session.
Thank you again .
Thank you very much Hani for sharing your opinion and information.
Regards
Hi Jabar,
I suggest a different approach. You could start off by outline a research question e.g.
"Can a neural network classify . . . etc"
"Is a decision tree better than a neural network for . . . .etc."
Then ask your students to turn this into testable/falsifiable hypothesis
The ask you students what should be measured to collect the evidence about the hypotheses
Then some good ideas about how to evaluate the ANNs should naturally emerge from this and the measures will be clearly useful tot he students. Nice little workshop activity which tye should enjoy.
performance of NN is dependent on which application its used for. For classification applications prediction and recall are good metrics.
Thank you very much James for great suggestion and help.
I will implement your suggestion in order to record the students response and contributions.
regards
- Badges
- Science topic