Hello sir,

Yes, I believe linear regression will show which independent variables (or a combination of variables) has maximum effect on the dependent variable (r values/co-efficent scores). I believe this tutorial link will clearly demonstrates and answer your queries. This was the most simple and to the point explanation that I could find.

Regards.

https://youtu.be/ZyruUhomEnQ

Regards.

@wati thanks for your answer. I have seen the video and many others too.

But my point is the dependent variable is not obtained or a score of all the independents. In my case the dependent variable is itself composed of all the variables that i am putting in the independent list during stepwise regression. Can i do this to chcek the amount of variance the R square in the dependent variable? Is this technically correct?

Soumen Sadhu -

First, your dependent variable should be the actual disease severity. You need to have some way of determining that, and the predictors are these different measures that you have. If you are saying that some kind of average of these six scores is defined to be the severity, and you want to know which set of up to three comes closest to the full measure as opposed to using all six, then I guess ideally a set of one to three would give all the information, and the information in the other three to five would all be part of what is already contained in the ideal set of variables. Collinearity would explain why some could be eliminated. The variables kept would contain all the information of those dropped, and hopefully more information than that. That would be the ideal, I suppose. However, likely some information would be left out, and perhaps like an omitted variable problem, perhaps this might substantially bias results. I'm not sure this is OK, but I guess you could try it.

However, I would not use a stepwise approach to finding the best set of variables. The variables, even when some are called 'independent variables,' actually influence one another, and the best set will not necessarily be able to be picked one at time. The order in which you consider them may influence which ones are chosen that way. One consideration might be to decide on some alternative sets of one to three variables to keep, based on your subject matter knowledge. You could then compare the performance of these alternative models for the sample you have by doing a "graphical residual analysis," plotting results for each alternative on the same scatterplot, for your sample. However, you do not want to overfit to that particular sample, so a "cross-validation" for each model would be helpful. In this case, since you are trying to reduce the number of variables used, perhaps overfitting to the sample may not be as big of a consideration as usual, but I'm not certain.

Plotting your data may be the best that can be done for making overall comparisons, as long as you do not try to glean too much information from a small sample, a problem that always needs to be considered.

It would be interesting if you would show a graphical residual analysis for this on this thread.

A problem may be whether the 'average' of those three measures truly is the actual disease severity. How do you define disease severity?

You probably know or suspect which measures are most important. Perhaps you might try ranking cases by the 'average' measure, and ranking them again by a composite measure using fewer measures, which you suspect might be about as good, and then compare those lists to see if and how they differ. You might even suspect that the reduced measure list does better on occasion, perhaps.

A lot may depend on an actual determination as to the definition of 'disease severity.'

I hope that I addressed the question that you wanted addressed.

Best wishes - Jim

Step-wise technique is an outdated one due to several drawbacks. It may not be able to handle overfitting and underfitting. Go with adjusted R square if variables are just 6. Check measures like AIC and BIC. Remove variables with high VIF.

As I noted above:

"Plotting your data may be the best that can be done for making overall comparisons, as long as you do not try to glean too much information from a small sample, a problem that always needs to be considered."

I noted that

1) You could compare alternative model results for a given sample on the same scatterplot using a graphical residual analysis, and

2) a cross-validation can help avoid overfitting to that one particular sample.

You could compare fits for the alternative models for one sample on one scatterplot, and then compare their fits for another sample on another scatterplot. This is likely far more informative than a few nebulous statistics.

A stepwise approach is not likely to find the best combination of predictors. The best combination may not come up as predictors may be eliminated based on how they perform in combination with other variables before you see them in combination with the ones with which they work best. Subject matter knowledge may be helpful. Other techniques may work, but may be foiled by the particular sample drawn before getting a proper trial.

The best combination of predictors, used in the best way is important. This is key in the following:

https://www.researchgate.net/publication/354854317_WHEN_WOULD_HETEROSCEDASTICITY_IN_REGRESSION_OCCUR.