It would make more sense to introduce the 'main' independent variable (i.e., the one whose effect you care most about testing) last, rather than first. It sounds like the idea is to test how much variance that variable accounts for over and above whatever might be accounted for by the other variables.

If you care about looking at the variance accounted by each variable in the model, you could also look at semipartial correlations.

You may use a correlation matrix.

try the following:

https://m.youtube.com/watch?v=-KSqxMvUgRk 

You should also probably mention what criterion you used in the variable selection procedure (AIC, adjusted r2, Mallows's Cp, etc). Different criteria may give different sequences of nested models (similarly to the LASSO path), and thus, may give different answers. The correlation of explanatory variables should also be studied to prevent multicolinearity (and thus to prevent selection of regression models with good prediction properties but with many strong signals non-significant).

Hello Gilad Sabo! The interpretation of R or adjusted R is not affected by the regression technique used (i.e., forward or stepwise) for variable selection. That is, forward or stepwise are used to select the variables that add significant variability to the statistical model. Thus, the R or Adjusted R determines the total variability contributed by the independent variables in the model. If you are using only one independent variable and such variable explains only 5% of the variability of the data, then the model does not explain well the data (the regression model does not fit the data). If the regression model has two independent variables the R or adjusted R will determine the total variability in the model that is explained by the two independent variables, say, adj R=.80, which indicates a good model fit. Individually, each variable can account for, say, .50 and .30 of the variability. If one variable accounts for .75 and the other only for.05, this second independent variable is not a good predictor or explanation for the dependent variable. I hope this helps.

Hello Gilad,

first: you write: "regression with five variables to predict one dependent variable". and than you name only 3: ABR, Age, Sex. I am confused.

A different approach/view:

your proposed model is: general knowledge score depends on ABR with covariates Age and Sex.

If you than look at models with less predictors, these model are biased. So it is of no use to inspect these models; they are misleading by default..

I advise: interpret the complete model with standardized variables ( BUT: do not standardize sex as it is a dichotomy).

You should report the complete model.

(Besides: moderation is operationalized with an  interaction (in order to moderate the relation between two variables). You did not include an interaction as far as can see in your post. You included covariates.)

Greetings, Pierre

Hi Pierre,

You are right, this is so basic,,,Interaction and moderation...

You are also right that I indicated that there were five independent variables but I didn't mention them all. So, the five predictors entered to the regression were age, sex (as dummy variable with values of 1 and 2),  annual book reading (ABR), years of education and income. The dependent variable was general knowledge.  

In the regression model (stepwise) only ABR and the covariates age and sex explained the variance in general knowledge scores. 

Are you suggesting to run the model again with z scores? I actually get the same results.

Thanks in advance,

Gilad

Hi Gilad,

About the predictors;

-I suppose you have measured age  in weeks or days. Not in reported years because when measured in reported years there is always an (not symmetric) error upwards and never downwards (for instance: a person who says she's 18 years old and has her birthday in 1 week actually is almost one year older then she says)

-you have 5 predictors, that means at least 60 cases (for 5 parameters and one intercept).

-I would code sex as 1 and 0, for the ease of interpretation.

The analysis:

-I would NOT use stepwise or forward or backward. YOU must decide on the model. Not some program. And again: incomplete models are biased.

-Indeed  I would use z-scores for all the interval variables. And yes in terms of explained variance and p-values the results are the same. But the interpretation is easier. And within the analysis you can compare effects.

-You write:"In the regression model (stepwise) only ABR and the covariates age and sex explained the variance in general knowledge scores. " I suppose that is decided by p-values. If the sample size doubles, the p-values change. So... how informative is a p-value?

greetings, Pierre

You did not mention the purpose of the model, is it to predict or explain? If explanation is the purpose, then Pierre is quite correct that variable selection is not appropriate. If prediction is the goal, then variable selection can be useful. I would not recommend stepwise, forward or back ward selection because of stability issues. The lasso is an improvement on that. I believe the information that you want is available in the regularization plot of lasso. All of this runs easily in R or SAS. For starting references and R code, Google Boos Adaptive Lasso R. Denis Boos has links to the papers and software.  Since you have some categorical variables you will need the Group Lasso algorithm which is also available in R, simply use Google to find the references and the appropriate R package, Best wishes.

Thanks David, 

I actually found a problem of multicollinearity in one of my IVs (VIFs values > 10 with two other IVs, rest of the IVs are good with VIF values are

Hi Gilad,

Your choice of dependent variable should be based on your objectives. But you should first run a correlation analysis to identify the significant variables before you continue with regression

"you should first run a correlation analysis to identify the significant variables" will decrease the degrees of freedom in a subsequent analysis and thereby increase the Type I error.

Dear Gilad

I suggest  first run a correlation analysis to identify the significant variables" Then  run regression with the selected variables. There is no peculiar way of interpreting forward regression. Except that you enter the variables one at a time and look at the result based on the significance of the variables.  You should also check the model adequacy as you enter the variables to check those that actually contributes to the variation experienced by the explanatory variable

Gilad, Ridge is a good choice. It has only a different penalty than lasso. I would recommend using elastic net in R. It also does lasso.There are many good references on elastic net. I particularly like Lander, Jared, R for Everyone. I've attached a link. Best wishes.

https://www.amazon.com/s/ref=nb_sb_noss_1?url=search-alias%3Daps&field-keywords=jared+lander