For time-series forecasting, I'm using an LSTM network. Are there any metrics that could be used to evaluate the forecasting model's generalization throughout the training phase, i.e., whether it is neither overfitting nor underfitting? To check that the network is not overfitted, for instance, we can look at both the training loss and validation loss curves. Can such overfitting or underfitting be detected using any tables?
Hi, the short answer is to keep monitoring the loss and val_loss values.
If the loss keeps decreasing below the val_loss while the val_loss value stops or starts to increase, this is a clear sign of overfitting.
The robustness (or generalizability) of a model is a measure of its successful application to data sets other than the one used for training and testing.
https://stats.stackexchange.com/questions/224014/by-which-ways-can-we-in-principle-evaluate-whether-a-model-succeeded-in-genera
Article Developing more generalizable prediction models from pooled ...
Comparison of the Validity and Generalizability of Machine Learning Algorithms for the Prediction of Energy Expenditure: Validation Study
Have a look at https://analyticsindiamag.com/a-guide-to-generalization-and-regularization-in-machine-learning/
Muhammad Ali Shima Shafiee Abdalraheem Ijjeh
Thank you so much for responding to my question. Yes, I believe the training loss and the validation loss curves provide a good way to visualize the generalization of the network.
Also, I have found many papers using training and validation loss scores, which is the final MSE value that the algorithm calculates while training the model.
- Badges
- Science topic