Hello Ayesha,

In addition to David Eugene Booth 's suggestions, please allow me to add some small points.

1. You haven't indicated in your query what specifically you're trying to find out about model 'reliability/validity.' What people frequently do when trying to determine how well a regression model works beyond the data set used to generate it is: (a) apply the model and its coefficients to a new data set (or held-out data set, as you appear to have); (b) correlate the observed DV values in the new data set with the estimated/predicted DV values from the model as applied; then (c) compare the original (model generation data set) R-squared with the squared correlation from step (b). To the extent that these are similar, you gain confidence that the model performs comparably when applied to different data, and therefore that it could possibly generalize to the target population. If the two values are quite different, then you have evidence that the model's performance is quite sample specific, and may not generalize to the population.

2. There's nothing in SEM (covariance based or PLS based) that precludes you from using categorical IVs. These can be recoded into dummy variates if nominal, or you can find software that will handle ordinal IVs.

3. Bootstrapping doesn't depend on whether IVs are continuous or categorical (as in #2). The aims of bootstrapping/resampling methods are more aligned with robust error estimates for model parameters than with model generalization, in my experience. So, I'm not sure that there's any "battle" between bootstrapping and cross-validation as methodological foes.

Finally, Professor Booth's intimation that, for just about any analytic method, there's likely more options readily available within the R system and its growing collection of libraries than any other statistical software platform, is correct.

Good luck with your work.

Since you are using a predictive regression model, I would look at this link:

https://b-ok.cc/book/5243562/0e8c4b

Honestly, I would prefer R instead of SPSS for a number of reasons that you can Google. However in your case R has clear routines to do all of this for you.

Assistance to a new R(free down load to all) user can be found here:

https://b-ok.cc/book/3380717/14956f

Best wishes, David Booth

Hello Ayesha,

In addition to David Eugene Booth 's suggestions, please allow me to add some small points.

1. You haven't indicated in your query what specifically you're trying to find out about model 'reliability/validity.' What people frequently do when trying to determine how well a regression model works beyond the data set used to generate it is: (a) apply the model and its coefficients to a new data set (or held-out data set, as you appear to have); (b) correlate the observed DV values in the new data set with the estimated/predicted DV values from the model as applied; then (c) compare the original (model generation data set) R-squared with the squared correlation from step (b). To the extent that these are similar, you gain confidence that the model performs comparably when applied to different data, and therefore that it could possibly generalize to the target population. If the two values are quite different, then you have evidence that the model's performance is quite sample specific, and may not generalize to the population.

2. There's nothing in SEM (covariance based or PLS based) that precludes you from using categorical IVs. These can be recoded into dummy variates if nominal, or you can find software that will handle ordinal IVs.

3. Bootstrapping doesn't depend on whether IVs are continuous or categorical (as in #2). The aims of bootstrapping/resampling methods are more aligned with robust error estimates for model parameters than with model generalization, in my experience. So, I'm not sure that there's any "battle" between bootstrapping and cross-validation as methodological foes.

Finally, Professor Booth's intimation that, for just about any analytic method, there's likely more options readily available within the R system and its growing collection of libraries than any other statistical software platform, is correct.

Good luck with your work.

Kindly check https://www.researchgate.net/publication/322887598_A_COMPARISON_OF_INTERNAL_VALIDATION_METHODS_FOR_VALIDATING_PREDICTIVE_MODELS_FOR_BINARY_DATA_WITH_RARE_EVENTS

Also check https://www.sciencedirect.com/topics/computer-science/split-validation-operator