Hi Zeinab,

There are a few possible tests that you may have employed that could be considered a "Wald Chi-square test" in this context. I'll provide an answer for the most likely tests you used. Since you are using R, I presume you used glm() to fit the logistic regression. It's possible that a different function from a different package uses different tests, so beware.

(Possibility 1): You are testing whether your model is different from the saturated model. The saturated model uses the observed outcome to measure the model fitness. Since the saturated model uses the observed data, its fit is necessarily perfect. You may test your logistic regression model against the saturated model by calculating the deviance, which is essentially a measure of how much "fitness" is lost between your model and the saturated model. The glm() function from the stats package calls this "residual deviance". Ideally, your logistic regression's fit would be very close to the fit of the saturated model. That would indicate that you have a simpler model that explains the distribution of your outcome almost as well as the observed outcome. Therefore, you would hope that your p-value is > 0.05, as that result implies your model's not statistically significantly different from the saturated model.

(Possibility 2): You are testing whether your model is different from the null model. The null model is the intercept-only parameter model. I presume you are testing against this model as you mentioned the intercept-only model in your question. Conversely to the saturated model, the null model has the worst possible fit. You may test your logistic regression model against the null model by calculating the null deviance (also called "null deviance" in glm()). Ideally, you would hope that your logistic regression model is much different from the null model. Therefore, a p-value < 0.05 is desired so that you may conclude that your model has a better fit than the null model.

To answer your questions specifically:

1. A p-value of 0.37 (if you are using a rejection region of < 0.05) indicates that your model is not statistically significantly different from the comparison model (either saturated or null; see above). Therefore, you would not conclude any difference between your model and the comparison model. If you are testing against the saturated model, then your model has a good fit. If you are testing against the null model, then your model does not have a good fit.

2. The Wald Chi-square test is a specific type of hypothesis test that estimates whether the difference between your estimate and an assumed null value is "unusually large" if the null hypothesis is true. This test assumes the null hypothesis is distributed as a chi-squared random variable (though not all Wald tests use chi-squared). Its magnitude is more general than in relation to the goodness-of-fit for your model. However, in this context, a higher value would indicate a lower probability that you would expect to see a difference in your model and the assumed model if the null hypothesis (equal model deviance) were true. Whether this indicates a better or worse fitting model depends on which of the two possibilities above relate to your model.

Two things to note for question 2: (a) the chi-square value doesn't provide any new information that the p-value doesn't already provide and (b) the test of deviances is a likelihood ratio-test. If you used a Wald test, then it may give different answers from what the default glm() function provides about model fitness.

I hope this helps.

1. No. 2 No. You might find the attached to be of interest. Best wishes, David Booth

Joshua Rutsohn Thank you so much. Yes I am using glm() in R. You touched on the point that was confusing me; there is not standard decision point on the test it depends on what are we comparing with (saturated or null models).

One thing is that I am not sure what is the model is comparing with (I though it is null by default).

For the LR test I used these:

#model0 is the intercept only model

model0

David Eugene Booth Thank you.

The LRT is a general test that compares two nested models. To get a p-value one would need the distribution of the LRT statistic under the restricted model. This distribution is usually not known but is can usually be approximated by a Chi² distribution with df = the number of restriced parameters (-> Wilk's theorem). This approximation is based on the fact that the shape of the log likelihood ratio approximates a paraboloid.

The Wald test is a special case of a LRT test using the paraboloidal approximationof the log likelihood ratio also to calculate the LRT statistic (the likelihood is evaluated only for the maximum under the unrestricted model; the likelihood under the restricted model is calculated based on this value and the Fisher information matrix (estimated from the Hessian at the maximum of the unrestricted likelihood). This can be done quickly for each single parameter or coefficient in the model. One can also use the Fisher Information to get the standard errors of the coefficients and the ratio beta_hat / se_[beta_hat] is the z-distributed (the square of it is chi²(1)-distributed.

Wald tests may not work good for small samples and in cases where a parameter value is close to a limit of the parameter space.

Apart from this:

David Eugene Booth is absolutely correct. Note that "Absence of evidence is not evidence of absence". It's easy to get a large p-value simply by using a smaller sample. You should not take this as evidence for anything (e.g. as evidance for a "good fit" of your null model). And a high Chi² value is just saying that the likelihoods are very different (see https://en.wikipedia.org/wiki/Likelihood-ratio_test#Definition to see that this is just a scaled difference of the log likelihoods). This may indicate that your sample provides sufficent information regarding the difference in the two models that are compared.

Jochen Wilhelm Alright I see.

Thank you , much appreciated.