Hello. I want to use a machine learning method for function regression in order to speed up metaheuristic methods for optimization. Given all these methods such as multilayer perceptrons, radial basis networks, suport vector regression, etc., there is a variance both in accuracy and in duration of training and evaluation. In some tests for checking accuracy and duration, others are fast in training and evaluation, but produce poor results, while others produce very accurate results but are very slow in training and evaluation. Others are somewhere in the middle. Should I choose accuracy over duration or the opposite? Or perhaps should I take the middle way?
Hi,
Most important is building a model that can be generalized effectively. Accuracy is fine but don't rely on it alone--you need to look at another aspects such as ROC, confusion matrix, etc. Time is not the whole story; what's the benefit of fast training process if the resulting model has low predictive capability?
BTW, MLP takes long time in many scenarios.
HTH.
Samer, PhD.
Dr Sarsam, I had the impression that ROC and confusion matrix are for classification methods. What I am talking about is prediction for real-valued functions, such as predicting the value of a house
One more thing, if you're mainly concerned in predicting a numeric value, you need to deal with a regression problem where correlation coefficient and RMSE are useful (rather than ROC and accuracy) when comparing several algorithms, as no one can tell which algorithm works perfectly for your task. Hence, compare several regression algorithms to figure this issue out.
Cheers,
Samer
- Badges
- Science topic