Hello,
I am working on SEM-PLS environment and I am testing the predictive relevance of two nested models analysed with the same data-set. In the second model I just added a predictor.
Variables are not normally distributed.
I would like to test if the difference in the R2 of the two models is significant. I got Bias-Corrected accelerated confidence intervals for R2 values in both models through bootstrapping procedure (5000 samples).
It is safe to say that if those confidence intervals do not overlap then the two R2 are different? Or should I account for the number of variables added before comparing the R2? How?
Also, I was thinking to just get the differences of R2 for each sample produced through the bootstrap procedure. Then construct the Bias corrected confidence interval on the distribution of those differences.
Do you have any suggections?
Thank you!
E.
There is no known distribution for R^2 so I have idea how to test it. Besides that you have two R^2 values. What would your bootstrap sample be?
This doesn't look very promising. Best, D. Booth
Hi David Eugene Booth
thank you for your reply.
I will get R2 values for each of the 5000 samples obtained through bootstrapping for both models. This will provide me with one distribution of 5000 R2 values for model 1 and 5000 R2 values for model 2.
Bias-Corrected Confidence intervals can then be created on the basis of those 5000 r2 values for each model.
My point is, can I jusrt check if the confidence intervals obtained overlaps to find if the R2 of the two models are statistically significant or not?
If not, could I just compute the difference between the 5000 thousand subsamples obtained for model 1 and model 2 through bootstrapping and then calculate the Bias corrected confidence intervals on the differences of those samples? Then if the confidence interval includes 0 means that there is no significance difference between those R2 (see below for example)
Model 1 R2 Model 2 R2 Difference
Sample 1 0.500 - 0.400 0.100
Sample 2 0.350 0.410 0.060
Sample 3 ................................................................................
Sample n
Sample 5000
While it is true that you can do as you suggest there is still no known test for comparing R^2 values. In other words what distribution should you use. An other way to put this is that you seem to think R^2 is a random variable. Some other people have thought this and worked on the problem. See the links:
https://stats.stackexchange.com/questions/130069/what-is-the-distribution-of-r2-in-linear-regression-under-the-null-hypothesis
https://www.google.com/search?q=Is+R%5E2+a+random+variable%3F&rlz=1C1CHBF_enUS847US847&oq=Is+R%5E2+a+random+variable%3F&aqs=chrome..69i57j33l2.19478j0j7&sourceid=chrome&ie=UTF-8
Best wishes, David Booth
I agree with David. There is no known distribution for R ^ 2. However, the logic tells me that if both models are significant (p-value