Pay attention to null hypothesis in Hosmer and Lemeshow test.

"The null hypothesis is that the observed and expected proportions are the same across all doses. The alternative hypothesis is that the observed and expected proportions are not the same. The Pearson chi-squared statistic is the sum of (observed – expected)^2/expected."

You were also interested in seeing the following link.

https://en.wikipedia.org/wiki/Hosmer%E2%80%93Lemeshow_test

Sample size guidelines like 10 observations per predictor are notoriously problematic and not a substitute for a priori power analysis. Put simply you can still have low statistical power even if these guidelines are met. With a total sample size of 50 you don't have a great deal of power to detect anything except very large effects. Furthermore your four predictors are likely correlated. (Guidelines like 10 observations per predictor are actually taken from research that ignores collinearity completely). This means that the effective sample size to detect the unique effects of each predictor is lower than 50 because these tests ignore shared variance between predictors. For instance if the correlation between a predictor and the other predictors is .5 then you'd have .25 shared variance and effective sample size .75 of the full N for that test.

So the reason for the lack of statistical significance may well simply be low statistical power. I don't think the Hosmer and Lemeshow test is relevant to this at all.

Hello Tishta Bachoo. I suggest that you look at Frank Harrell's checklist (link given below), and pay attention especially to the Overfitting and lack of model validation section.

• https://discourse.datamethods.org/t/author-checklist/3407

I'll highlight a couple of points.

In your case, 38 of the 50 Ss had IR expertise and 12 did not. So the effective sample size = 12.

Applying the 10:1 rule to your situation would allow just one (continuous or dichotomous) explanatory variable. And according to the 20:1 rule, your effective sample size is too small to support modeling at all.

Finally, bear in mind that the 20:1 (or 10:1) rule of thumb is primarily about avoiding over-fitting of the model, whereas power analysis & sample size estimation is about ensuring a large enough sample size to ensure a given level power to detect the smallest effect size of interest. I suspect that the sample size yielded by power analysis will often be large enough to meet the 20:1 rule, but not necessarily so, as they address two related but different issues.

HTH.

PS- As Thom Baguley noted, rules of thumb like 10:1 or 20:1 can be problematic, depending on the particular circumstances.

Dear Tishta Bachoo ,

Dividing the subjects into deciles based on the predicted probabilities, the Hosmer and Lameshow X2 (1) statistic with Sig 0.05 computed from observed and expected frequencies shows that the model with a predictor provides a better fit than the null model; and concurrently predictor variable(s) should be statistically significant at 5 percent level.

Suggestion: Two of your predictors (Role and Qualification) seem to be close to becoming significant at 5 percent level. Exclude (deactivate) the constant term in your function/model, re-run your estimation and see if the respective predictor variables (role and qualifications) do not become statistically significant.

Hello Tishta,

The two results represent tests of different hypotheses, and therefore need not agree with one another.

The H-L test evaluates whether the performance of the final model is consistent across levels of the independent variable (useful when one or more of the IVs is continuous in form). In other words, does the model work coherently across the range of the IV (this is correctly implied by Abolfazi's response, above).

The test of individual IVs is simply a matter of whether including that IV yields a non-zero regression coefficient (which may correctly be construed as, does the IV contribute to the explanatory power of the model).

So, there's no reason to expect that the two tests will or must agree for any given data set.

Good luck with your work.

Thank you so much everyone for your insights :)

In addition to David Morse excellent answer I note that the H-L test is no longer recommended. A quote from the Wikipedia entry is given here:

"The Hosmer–Lemeshow test has limitations. Harrell describes several:[3]

"The Hosmer-Lemeshow test is for overall calibration error, not for any particular lack of fit such as quadratic effects. It does not properly take overfitting into account, is arbitrary to choice of bins and method of computing quantiles, and often has power that is too low."

"For these reasons the Hosmer-Lemeshow test is no longer recommended. Hosmer et al have a better one d.f. omnibus test of fit, implemented in the R rms package residuals.lrm function."

"But I recommend specifying the model to make it more likely to fit up front (especially with regard to relaxing linearity assumptions using regression splines) and using the bootstrap to estimate overfitting and to get an overfitting-corrected high-resolution smooth calibration curve to check absolute accuracy. These are done using the R rms package."

Other alternatives have been developed to address the limitations of the Hosmer-Lemeshow test. These include the Osius-Rojek test and the Stukel test.[4" "