Can you explain it or recommend some papers?
(1) For classification problem, we can use recall rate, precision rate and F1 score, AUC, balanced accuracy(although I don't know what is mean/ max balanced accuracy) in same time to compare different classifiers.
but I haven't see any paper to evaluate different numerical prediction models (forgive my ignorance).
(2) I only know r-squared, MAE, MAPE and RMSE could be used to assess models, but how to combine them to evaluate in a certain case, should add any other metrics?
(3) Could you help explain max/mean balanced accuracy, mean/max recall, mean/max accuracy, mean/max AUC, mean/max F1 if you are free. Why use mean or max, aren't these values specific values? Why are they divided into mean and max?
Dear Ling you can see the below link to study more about forecasting evaluation metrics and indices also you can find cons and pros of each metrics and proper metric for your specific data type :
https://www.analyticsvidhya.com/blog/2021/10/evaluation-metric-for-regression-models/
hope it helps you .
Good luck
There are many different metrics that can be used to evaluate numerical prediction models. Some commonly used metrics include R-squared, mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). Each of these metrics has its own strengths and weaknesses, and the choice of metric(s) to use will depend on the specific goals of the analysis.
One paper that discusses evaluating numerical prediction models is "PM10 and PM2.5 real-time prediction models using an interpolated convolutional neural network" by Chae et al. ¹. Another paper that discusses evaluating prediction models more generally is "Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA)" by Snell et al. ².
Regarding your question about mean/max balanced accuracy, recall, accuracy, AUC, and F1 score: these are all metrics used to evaluate classification models. The choice between using the mean or max value of these metrics will depend on the specific goals of the analysis. For example, if you are interested in the average performance of a model across multiple classes or multiple runs, you might use the mean value. If you are interested in the best possible performance of a model, you might use the max value.
(1) PM10 and PM2.5 real-time prediction models using an ... - Nature. https://www.nature.com/articles/s41598-021-91253-9.
(2) Transparent reporting of multivariable prediction models for individual .... https://www.bmj.com/content/381/bmj-2022-073538.
(3) [1905.11744] Evaluating time series forecasting models: An empirical .... https://arxiv.org/abs/1905.11744.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python