in a machine learning, there is a possibility of overfitting training data and carrying the noise of that data through to the test set, that give inaccurate generalizations.
What are methods available to avoid overfitting, other than below methods :
1- Keep the model simpler: remove some of the noise in the training data.
2- Use cross-validation techniques such as k-folds cross-validation.
3- Use regularization techniques such as LASSO
Thanks
All of these methods are useful, and if you have a large enough and well-sampled dataset there's a good argument for throwing in the kitchen sink and trusting in regularisation and cross-validation to keep you on track. But if the data is sparser (and few of us really have access to BIG data), then often the most useful thing you can is to form some intuition about the problem and the structure of the data that will allow you to simplify the problem.
Can you guess some features that are likely to be highly predictive? If so code them explicitly in your dataset. If you're using time series then look for trends and seasonality etc. before you start crunching large numbers. If you're dealing with image analysis you might already know what type of features are likely to be important and you could recode using established dictionaries.
Consider what pre-processing steps might be useful (whitening images, differencing time series etc.). Depending on the type of data involved there may be useful transformations that you could apply before fitting a model (Fourier, wavelet, cosine transform etc.)
Basically, if you have some idea what you're trying to find, the you've a much better chance of finding it (provided it's actually there).
Dear,
if your model performances are good (AUC > 0.9) when tested using data different from those used for training this will demonstrate the ability of generalization, thus a poor generalization and good results on training data confirm the overfitting. tuning the training of your model to avoid this problem depends on the model itself. hope this help
There's no way of ``ensuring''-especially as long as ``overfitting'' isn't defined.
A machine learning model (indeed any model) isn't an end of itself-it's a means for describing ``something''. That ``something'' is what's relevant.
If you know that (or want to test whether) the data are described by a linear relation, three points and more provide consistency checks; if you know that (or want to test whether) the data lie on a conic, six points and more provide consistency checks. And so on.
As far as i know, there were currently at least 4 methods to indicate whether your model performs in the right way of not overfitting:
1. AUC curve as Dairi Abdelkader mentioned.
2. K-fold cross validations
3. Test on another independent dataset
4. Augmentation of the whole dataset for testing.
You need to test the error difference on the trained data with that of untrained test data.. As soon as the error starts rising in the untrained data. The model's stability decreases. You can achieve that by plotting your error results and visually deciding when to stop its further training . Usually we do it on batch steps. Say 1-200 epoch of training you check its performance on test data. Thereafter, as per the performance of the model you can decide to further train it from 200 to 400 epochs or stop there itself.
Try writing log file for error reported in each epochs and then comparing it batch-wise.
However, if your are looking for more scientific/complex way... k-fold cross validation will do the trick.
- Badges
- Science topic