Machine learning regression algorithm is widely used in remote sensing community. But when the dataset only contains small sample number, how can we use the machine learning regression algorithm? And how to get resonable and robust regression algorithm?
For small dataset, I recommend using cross-validation, rather than separating one training set and one validating set. In this case it is possible to use the entire database for training, without compromising the validation.
I agree with @Roberto Filgueiras. In most cases, cross validation and 1 hold method should help in building a model with number of iterations. There is one more technique to generate new data from existing samples using estimation method. Its not always advisable to be used but, sometimes it helps. I have attached a worked out document. Hope it helps.
In case of small sample set, you may use Random Forest Regression algorithm, which will lead to accurate estimate as well......
For small samples use Leave One Out Cross Validation (LOOCV). This is a special case of cross-validation where the number of folds is equal to the number of samples. In each cycle one sample is used in validation and the other in training. The operation is repeated n times where n = number of samples. Regression performance is the average of the indicator used (R2, RMSE) etc.
I agree with what was stated above. Also, regularized regression (e.g. ridge regression) or a regression with a cost function (e.g. SVM) perform well on small data sets. These techniques help with collinearity, over fitting, and less sensitive to the data distribution.
For any possible data spiliting prior to modeling, consider the possibility of artificial creation of covariate shift in your data set then you can at least calculate uncertainity of your model for data out of your training set.
In line with the response from Trevan Flynn, SVM, RT, RF can do better. in the case where your dataset is small and you split it, it is a good idea to try normalization before using it for regression with SVM, or RT or RF. The results would be amazing.
- Badges
- Science topic