Marrium Sultan Dar -

I do not suggest using hypothesis tests. If you want to know how well your model fits your data, you can research "graphical residual analysis." That will show you how well your model fits the sample you have. You might even overfit to that specific sample, so you could research "cross-validation" to address that issue.

A graphical residual analysis is a scatterplot with predicted-y on the x-axis and estimated residuals on the y-axis, or you could account for the h-values and use studentized residuals. A graphical residual analysis is a step in investigating heteroscedasticity, which is also part of model fit.

The problem with an hypothesis test is that even when you account for effect size, which is essential, you are still looking at a "yes" or "no" "answer" when you generally need to estimate degree. Estimation of parameters may be better, but here "A picture is worth a thousand words." Graphs can be very informative.

Cheers - Jim

Hello Marium,

1. Yes. Unless all of your candidate IVs are unrelated to one another, the determination of statistical significance for a selected IV can be different for the bivariate model case (one IV, one DV) than for the multiple regression model case (two or more IVs, one DV).

2. In multiple regression, the test of significance for a selected IV is in context; viz., given all of the other IVs in the model.

In your field, this situation is not unlike studies in which authors report "raw" or "crude" odds and then "adjusted" odds. The first is for single IVs, the latter is for multiple IVs simultaneously entered into a model.

Finally, James Knaub's advice is sound: It's always good practice to inspect data, both before (to determine suitability and possible issues) and after applying a model (to determine possible problems with the model).

Good luck with your work.

You shall have at least 10 persons per independent variable. Some suggest more.

Sample size adequacy is better related to the attainment of adequately low estimated variance of the prediction error. That will vary for individual predictions, especially when you account for heteroscedasticity in regression. (For finite populations, one could also consider predicted totals as in the following, for example: https://www.researchgate.net/publication/261947825_Projected_Variance_for_the_Model-based_Classical_Ratio_Estimator_Estimating_Sample_Size_Requirements.)

Thanks a lot James R Knaub, David Morse and Béatrice Marianne Ewalds-Kvist!!

Your answers helped me a lot and I will surely follow your advice.👍🏻