I make statistic linear model with spss or/with matlab. I wanna to cross validate this model but i have 2 problem.
at first, suggest me a way in spss or much better to say in matlab to show me general model with selected descriptor and a way to calculate the predicted data with this model to use in Cross validation?
but the second problem that is more important, how to cross validate this model with leave one out method (LOOCV)?
LOO methods was out of date. It was developed in the age we have no computer power.
I developed the " k-fold cross validation for small sample " method. It is very helpful.
You have the Phirosopher's stone.
StackOverflow has some examples that may help.
You will need to step away from the GUI interface in SPSS and go back to the old command line/batch command format (which is much faster if you're doing repeated tasks anyway.) Also, obviously you don't need to create artificial data as the SPSS example shows in the first half of the sample code.
http://stackoverflow.com/questions/24216574/how-to-do-leave-one-out-cross-validation-in-spss
http://stackoverflow.com/questions/15433666/leave-one-out-cross-validation-algorithm-in-matlab
Se also: http://au.mathworks.com/help/stats/crossval.html
I have some MATLAB functions written by the group to carry out different types of cross-validation (including leave-one-out) and predict with different regression and classification methods. What method are you using?
Hi dear Matteo
is your question about my regression method?
Im using Stepwise linear regression.
Hi Mohamad,
so you are doing multiple linear regression and using stepwise to select descriptors/variables, correct?
If this is the case, we have made a toolbox for MATLAB to carry out variable selection based on the RSR method. The toolbox implements multiple linear regression and principal component regression. I attach the toolbox and the paper in case of interest to you.
To use the toolbox, just type START_RSR to start the gui for the creation of the options. To do leave-one-out, just set Number of cv groups to be equal to the number of samples in your dataset. Anyway, there is a html help file for the toolbox, but if you get into troubles, just write to me.
I hope this helps.
Hi, Mohamed
I was user of SPSS, but am user of JMP.
SPSS support a leave one out method for the discriminant analysis. However, LOO is not helpful for you. I developed the new method such as k-cross validation for small sample.
And I propose the best model with minimum mean error rates in the validation sample.
See my two papers published in 2014. These papers deal with the discriminant analysis.
You can apply this idea for the regression model
Hi, Mohamed
See this paper "A comparison of bootstrap methods and an adjusted bootstrap approach for estimating prediction error in microarray classification" Wenyu Jiang & Richard Simon
As an aside, you may want to reconsider the stepwise approach to regression, as it can be problematic.
http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/
Hi David
would you like explain more, i cant understand your meaning.
Hi, Akbari
I think you need not to use LOO or cross-validation for your regression model because you are not statistician.
At first, you try to use the stepwise method to build the good regression model.
See two files that are Yavuz and Diego questions and answer.
Hi, Dear Prof
In QSAR modelling validation is critical step and is essentially for our model to be sure that our model is predictive. this reason demonstrated in professor Alexander Tropsha (Be ware of q2) paper.
best,
mohamad
Hi,
Did you tell me the title "professor Alexander Tropsha (Be ware of q2) paper"?
Hi,
Sorry, may be i type it incorrectly. I attached the links.
http://onlinelibrary.wiley.com/doi/10.1002/qsar.200390007/abstract
http://www.sciencedirect.com/science/article/pii/S1093326301001231
LOO methods was out of date. It was developed in the age we have no computer power.
I developed the " k-fold cross validation for small sample " method. It is very helpful.
You have the Phirosopher's stone.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS