Not sure that I understand the reason, but I think the strata option in the function boot (in the package boot) might be what you are after. Do do you mean run the same model with the same seed and the same data 100 times?

I need to run the model 100 times to find it's average prediction accuracy and robustness to data changes. For each run, the dataset will be randomly split into a training subset and a testing subset. The training and testing error will be calculated for each and then averaged.

Worth googling "cross-validation."

# CV using boot

library(boot)

# Cost function for a binary classifier suggested by boot package

# (otherwise MSE is the default)

cost 0.5)

# LOOCV (accuracy)

1-cv.glm(dat,model,cost=cost)$delta[1]

# OUTPUT: [1] 0.79

# K-fold CV K=10 (accuracy)

1-cv.glm(dat,model,K=10,cost=cost)$delta[1]

# OUTPUT: [1] 0.79

Is cross validation same as running a model 100 times by changing the split?

@Daniel Thank you, I got the idea. I don't actually need to run the model 100 times, all I need is to create subsets by K-fold cross validation where K value will be 100 and I will get the required training and testing errors.

Just one question, as per the statement that I quoted from a paper " run the model 100 times to find it's average prediction accuracy and robustness to data changes. For each run, the dataset will be randomly split into a training subset and a testing subset. The training and testing error will be calculated for each and then averaged."

It's splitting the whole randomly into 70% and 30% subsets. Just changing the individual observations.

Can this be done by cross validation as well.

My software does something like that automatically.

www.lerenisplezant.be/fitting.htm