In Amos, I am explaining a dependent construct with one exogenous construct and get a squared multiple correlation for the dependent of .48. I then add a second exogenous construct as an additional predictor and the squared multiple correlation for the dependent falls to .37. How is that possible? When I do a normal regression, the R2 goes up, as it should.
I have asked two local experts here that could not help. Any help here? Thanks in advance!
Hi Klaus, I don't know anything about Amos, but my guess would be that it has to do with what kind of R2 you are looking at.
The classical R2 measures how well your model fits the data it was trained on. This is sometimes called "calibrated" explained variance and, as you say, it can only go up or remain the same by adding additional explanatory variables.
However, this is not true for another type of R2 - the one that measures how well your model generalizes to new data (so called "validated" explained variance). This can be obtained by using the model on a new dataset or by leaving out part of the original dataset (cross-validation) for this purpose. In this case, it is theoretically possible that the R2 decreases by adding a new variable, if the second variable does not bring additional information relevant to explaining the new data (e.g., it might just fit noise in the original dataset rather then some underlying trend).
My guess would be that your software by default runs some kind of cross-validation in your model, and therefore the R2 you get is in fact the validated explained variance. If the value go down by adding the second variable it might indicate that the new variable does not provide useful information for predicting new data but instead result in overfitting.
- Badges
- Science method
- Similar topics
- Methodology