Hi there, ML experts!
What are the various ML evaluation method available to evaluate the effectiveness of the model. In other words, evaluating the ML model performance.
Dear Selamat,
In order to evaluate the machine learning models, you will have to know the basic performance metrics of models. For example, accuracy, precision, recall, F1-score, or AUC values are important measures for classifiers. Following this, you are also requested to go through other supervised and unsupervised machine learning algorithms.
Hi Siti,
if you have available ground truth, then you can calculate measures such as accuracy, precision, recall, specificity, F score etc.
You can also look here: Thesis Methods for Engineering Symbolic Human Behaviour Models for ...
(page 195 (Performance evaluation) discusses some metrics about evaluating the performance of models).
You could also compare different models to see which is more suitable for the given data. When comparing two models, depending on the type of data, you can use different statistical test. You can find a guide about the suitable statistical tests here: Article How to select appropriate statistical test?
(there is a chart that shows which tests are appropriate given the data).
You could also calculate the model likelihood (i.e. how good the model fits the data).
The question is not clear! Can you please elaborate the topics? There are a lots of machine learning technique. You should name the exact technique of your interest? Thank you.
If you want to perform regression-based modeling studies, then the measure of regression co-efficient R2 (for instance, R2 > 0.5) and cross-validated co-efficient Q2 (for instance, Q2 > 0.5) may be used as indicators of the model predictivity. There are also other validational parameters (linked attached).
https://www.sciencedirect.com/science/article/pii/S1093326301001231
http://onlinelibrary.wiley.com/doi/10.1002/qsar.200710043/abstract
https://www.sciencedirect.com/science/article/pii/S0223523417304026?via%3Dihub
https://www.sciencedirect.com/science/article/pii/S0960894X16310964?via%3Dihub
Thanks.
Dear Selamat,
In order to evaluate the machine learning models, you will have to know the basic performance metrics of models. For example, accuracy, precision, recall, F1-score, or AUC values are important measures for classifiers. Following this, you are also requested to go through other supervised and unsupervised machine learning algorithms.
Dear Siti Aishah Md Selamat,
Look the link, maybe useful.
https://machinelearningmastery.com/how-to-evaluate-machine-learning-algorithms/
https://www.slideshare.net/AliceZheng3/evaluating-machine-learning-models-a-beginners-guide
Regards, Shafagat
Thank you for the prompt replies Kristina Yordanova, Sk. Abdul Amin, Sobhan Sarkar and Shafagat Mahmudova! I will check out the information provided accordingly. Cheers.
Check the following paper:
EVALUATION: FROM PRECISION, RECALL AND F-MEASURE TO ROC, INFORMEDNESS, MARKEDNESS & CORRELATION https://bioinfopublication.org/files/articles/2_1_1_JMLT.pdf
Thank you Mohammed Elmogy for sharing! Will check it out. Cheers.
Evaluating the performance of machine learning based classification models, we have to know the basic performance metrics. For example, RMSE, Kappa statistic, classification accuracy, tp-rate (or recall), fp-rate, precision, F1-measure or AUC values are important measures for the classifiers.
Here 20 topics from the 296 articles which have words ti=(Evaluat* Machin* Learning*) in their titles. Each topic is represented by words and phrases from these articles.
If you have any impressions about the topics and terms, please, let me know.
In brief, there are 7 important model evaluation metrics which are:
An interesting article discussing these metrics can be found in the following page:
https://www.analyticsvidhya.com/blog/2016/02/7-important-model-evaluation-error-metrics/
Thank you for your informative post.
For categorical variables with more than two potential values, how are their accuracy measures and F-scores calculated?
I have a dataset with variables (Population class, building type, Total floors) Building Type with possible values (Residential, commercial, Industry, Special Buildings), population class (High, MED, LOW) and the total floor is a numerical variable with values ranging from 1 to 35. After training the data I wanted to predict the "population class". I applied SVM on the datasets. How are the accuracy measures and F-scores calculated for my case? Is accuracy measure and F-Score a good metric for a categorical variable with values more than one? Am I doing the correct thing by evaluating the classification of the categorical variable (population class) with more than two potential values (High, MED, LOW)? What if any variable is an ordinal variable should the same metric and classification algorithms are applied to predict which are applied to binary variables?
Please I also need to know if a mean-square-error evaluation metric can be used with unsupervised machine learning algorithms?
Yes you can use mse in unsupervised learning. Just log in this your question in Google, you will get a whole lots of performance metrics, that can be applied in such situation.
- Badges
- Science topic