In this case clearly you are selecting significant variables relatively from each block and independently from other blocks. Hence when these variables are presented in the final model you cannot make conclusions or comparisons across above blocks because you have ignored the effect of variables across blocks, for predict response in the model.

This means that say you have selected some variables in each block as,

                  from block 1 n1 variables out of N1 total variables

                  from block 2 n2 variables out of N2 total variables

                  from block 3 n3 variables out of N3 total variables

                  etc...

here in block 1 you are eliminating (N1-n1) number of variables without checking the effect of those variables towards the response in the presence of variables from other blocks! But in the presence of those variables these eliminated variables might show some significance on response, that you cannot confirm without checking. So be sure to overcome this problem.

I hope this helps, good luck.

Hello Bonnie,'You can Please go through herewith the attached xls files in this it is explained very clearly and i thought this may helpful to you 

Selection of variables with small data sets poses many problems. In the context of the current discussion, it is specifically instability. See the Springer text books by Harrell (2001) and myself (2009).

My position is that any attempt to include external knowledge is useful. External knowledge may for example be to limit the set of potential variables for inclusion; or make a hierarchy to clarify what is more or less confirmatory, and what is exploratory (where surprising discoveries may simply represent noise ..). External knowledge may also include the definition of blocks of variables as suggested (clinical, psychological, demographics), although I cannot judge how much this really helps to stabilize the selection results.

Small sample size makes that we should anyway be very cautious in our interpretations. For stable selection I would recommend at least 50 observations per variable, while fitting a fully pre-specified model might require only 10 observations per variable; in any case more than 3.1 (80/26) as in the present study.

I recommend against using any stepwise regression methods, including the Backward Elimination. Such methods have theoretical flaws, and they take the researcher's decision making out. You need to be able to see and to evaluate a second best model and a third best model, ... etc.

Consider first checking for outliers and for high influence data values, followed by checking for multicollinearity. Then only start considering "all possible regression" by selecting candidate models with excellent fit (high adjusted R square, low MSE, low Mallow's Cp). Then obtain for each of your candidate models the PRESS statistic.

Now you have a few excellent models for fitting, and you can choose the model that is also excellent for prediction. The resulting model(s) will have no outliers or data values with negative impact on the regression, and you will have made sure that there is no serious multicollinearity. Backward Elimination cannot do such "thinking".

Hello Martin,

This text explains PRESS in detail:

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470542810.html

The regression model with the lowest PRESS is the best candidate model to be used for prediction (among the excellent regression candidate models based on fitting criteria).

definition only:  http://en.wikipedia.org/wiki/PRESS_statistic

SAS :  https://v8doc.sas.com/sashtml/stat/chap55/sect6.htm

R:  http://stevencarlislewalker.wordpress.com/2013/06/18/calculating-the-press-statistic-in-r/

SPSS (unclear): http://www-01.ibm.com/support/docview.wss?uid=swg21479373

In the past, and we did not have access to fast computers, it was a standard practice to use data splitting to validate the regression model (for prediction). You would split the data set into two halves, and then you would use the top half to obtain a fitted regression line, by which you predict the y values in the second half of the data.

These days, having fast computers, we can do n data splitting to obtain a prediction Sun of Squares Error (PRESS).

X     Y

x1  Y1

X2  Y2

.

.

.

Xn   Yn

step1: remove x1 and y1. Fit a regression with the remaining n-1 values. Predict y1 with this model. Obtain the residual y1-y1(without having used (x1,y1)). This is the first PRESS residual. r(1).

step 2: put (x1, y1) back in the data set. remove (x2, y2). Repeat similar to what you have done in step 1, and obtain r(2).

.

.

.

step n:  Remove (xn, yn). Obtain r(n).

PRESS= sum of the squared R(i) values. It is a SSE that measures how stable a model is for prediction (with new data values). PRESS> SS Error.

In SAS: In the model statement:

model y=x/cli p;

this will trigger to give the PRESS statistic.

My rule of thumb for multi linear regression using the forewards and backwards approaches is to have at least 5 observations per variable in the predictive equation. This means that you would not want to have more than 16 variables in your predictive equation. As you can lump the data into three broad classes, you probably would not want more than three variables. With 80 cases, this should work ok.  In addtion, I usually preceed that analysis with a correlation analysis and then make sure I don't have variables in my final equation that are higly correlated. I see that your approach of separating the variables into groups first is doing the same. The problem with separating them and making a selection before the multi linear regression analysis is that you can not be sure which of the variables from each class will work best in combination with variables from other clases. Once you have done a regression  run or two, and know which variables work best together, it is a bit easier to justify using one varialbe from a class and dropping the others.  I accept some of the previous comments that this approach has some flaws, but I also maintain that it is a good way to get a fairly quick understanding of relationships, or lack of relationships in a farily complex data set.

Using pairwise correlations may or may not reveal the presence of multicollinearity, William.  None of the stepwise regression procedures is geared for identifying prediction models.

Have you considered trying Principal Components regression or Partial Least Squares regression instead of MLR ?

You might also try structure equation modelling