You can see this article, it may help you.

https://machinelearningmastery.com/overfitting-machine-learning-models/#:~:text=We%20can%20identify%20if%20a,on%20a%20holdout%20test%20dataset.

Many Thanks, Fatima Nadeem Al-Aswadi for your answer.

Actually, I read this article and many others about overfitting.

My main point is

If I have 2 models

the first training loss is 0.123 and the test loss is 0.234 the difference is 0.111

the second training loss is 0.023 and the test loss is 0.034 the difference is 0.011

Are these models still suffer from overfitting and if yes. Why?

Which one is better and why?

Many thanks again.

To reduce overfitting is to lower the capacity of the model to memorize the training data

Dear @Ahmed,

you need to draw the carve between train and test ( or differences) for many iterations. Then if the training results keep up while the test results keep down that means it’s overfitting.

Hoping this is clear to you.

Good luck

Dear @Fatima,

You can visit this site for more details about the overfitting in term of training and testing data and how to deal with and good luck

https://towardsdatascience.com/handling-overfitting-in-deep-learning-models-c760ee047c6e

for deep learning it is best to increase your training data set to avoid or decrease overfitting , trying autoencoder or data augmentation (GAN) would be useful in this case, Otherwise you can try ensembling the learning to get a more stable model

High dimensional feature vector may lead to overfitting. Reduce the feature vector dimension through RFE(Recursive Feature Elimination with fixed number of features) and use ridge classifier model(Ridge regression model).

Dear Enas A. Raheem Swainson Sujana Daniel Reema Ahmad Fatima Nadeem Al-Aswadi

Thanks for sharing your answers, I really appreciate it.

All your answers are solutions to the overfitting problem. But I am looking at the criteria to detect overfitting and confirm that this model is overfitting the data while the other model is just fine.

For example, in the attached image,

Say model1 (the brown line represents the test loss )

Model 2 (the red line represents the test loss)

and say that the same training loss (blue line)

The two models stopped training when the training and testing losses started to diverge.

If the difference between the train and test losses for model 1 (brown) is 0.05

If the difference between the train and test losses for model 2 (blue) is 0.1

Let's start with this question

Which model is better? and why?

Dear Ahmed,

• Both models are overfitting, however, the answer of your question : which one is overfitting more? It’s Red model. Note that when the model is very complex, the gap between training and generalization/test error is very high. This is state of overfitting

More details can found here: https://vitalflux.com/model-complexity-overfitting-in-machine-learning/

Preprint Overfitting Mechanism and Avoidance in Deep Neural Networks

There are a few ways to identify if a deep learning model is overfitting the dataset.

Most probably if the validation loss is significantly higher than the training loss, it may indicate that the model is overfitting the training data.