Two PLS regression models are developed from a set of spectral calibration data. For each of the model RMSE of cross validation (RMSECV) and RMSE of prediction (RMSEP) is calculated. Which model (out of these two) should finally be chosen (i.e. model with lowest RMSECV or RMSEP, or somewhare in between)?
I would suggest you try variable selection to choose your model. I don't know what your spectroscopy method is but I assume the spectra are continuous. Thus lasso may be appropriate. See the links below. Stay away from stepwise. Best wishes.
https://search.yahoo.com/yhs/search?p=variable+selection+in+partial+least+squares&ei=UTF-8&hspart=mozilla&hsimp=yhs-002
http://www.amazon.com/gp/product/0321888030/ref=s9_simh_gw_g14_i1_r?ie=UTF8&fpl=fresh&pf_rd_m=ATVPDKIKX0DER&pf_rd_s=desktop-1&pf_rd_r=0XDHXF6W5RFZA99FXQ90&pf_rd_t=36701&pf_rd_p=2079475242&pf_rd_i=desktop
http://www4.stat.ncsu.edu/~boos/var.select/lasso.adaptive.html
http://www.sciencedirect.com/science/article/pii/S0169743912001542
Dear David Eugene Booth, Thank you for your suggestions, but my question is totally different. Its about the choice of RMSECV or RMSEP on which to select a PLSR model. I understand the variable selection approach (including GA). But what should be fitness measure for a model (RMSECV or RMSEP)?
It is to much to hope for that one statistic will suffice for all uses in all situations. If your question had an answer in the sense you pose it, only one statistic would be discussed and all the others would have fallen by the wayside. Do all regressions have the same purpose? See the attached paper.
Biswanath Mahanty RMSECV is used to construct the model and RMSEP is used to vet the model against "new" data that the model has not seen. RMSEP should be used if "new" data that the model was not build using, is used to predict. If you are just trying to explain the data and are not predicting new, unseen data then use RMSECV. RMSEP will always be greater than RMSECV.
- Badges
- Science topic
- Similar topics
- Biological Science
- Biochemistry
- Biochemical Techniques
- Analytical Chemistry
- pH