I am running an MLP to estimate an output give 3 variables and 1600 observations. I resample the data into 10 bags and in each bag, I performed 10-fold cross validation. However, when I average the results of the 10 bags, the outcome is worse than without doing bootstrap aggregating. So I am curious whether bootstrap aggregating is an useful method for improving my predictor or not.
Yes, bootstrap aggregating will help you in your study. Bagging will introduce more robustness.
Bootstrap aggregation improves prediction and classification especially for unstable procedures (when changing a little bit the training data leads to substantial changes in test results). For stable procedures, benefit is not so important.
Also, you may try to increase the number of bags, and for each bag to train on all the bag data and test on the out-of-bag data, so that you will have more data for training than when you do 10 fold cross validation in each bag.
If you want more information about bootstrap aggregation you can look at the chapter "bagging" in Friedman book: "The Element of Statistical Learning".
I hope that this will help.
- Badges
- Science topic