The "validity" of your p-value is ok when the model assumptions are plausible and when the experiment was performed to provide suitable data.

Depending on the size of the effect and the sample size it can happen that the probability of more extreme test statistics under the tested hypothesis becomes really tiny. In such cases minor deviations between assumptions and reality have a considerable relative effect on the p-values. Therefore, it usually does not make much sense to numerically differentiate tiny p-values. It is thus ok and recommended to simply report p

This is a problem faced often in analysis. There is never a guarantee that two tests will agree. So what does one do if they give conflicting results? I believe that domain expertise should be invoked to decide which way to go. If you have intuitive reasons to expect one of two results, then go ahead and believe it.

In your particular case, deviance test is perhaps the most commonly used. I would go ahead and believe the model. See what predictions it gives. If they are 'reasonable' you are in good shape.

Kalpani

How many profiles were in your data?

Deviance and Pearson chi square statistics might be affected easily by the number of profiles and sample size especially when you have a continous variable in your model.

You could try including interactions or other higher order terms into the model to increase goodness of fit. Another solution might be changing the link function to cloglog or else.

What is the value of an Rsquared and AUC you obtained?

Edit: Influential observations might also affect goodness of fit.

It is better to avoid using chi-square tests in that case.

Even a small deviation from the perfect fit, especially if the sample is large, may give significant tests.

Goodness of fit indicators are mostly ignored in biostatistics. Models never fit.

1) Quantitative variables almost never have linear effects. Ideally, it should be close to linear, but, in my opinion, as long as it has a monotone link, it's good enough.

2) The model is assumed to be additive (without interactions). But, this doesn't fit to reality where there are almost always some interactions even if they are not significant.

The question is not : Does it really fit ?

The question are : How far is it from fitting ? Where does it fit not ?

For that, descriptive statistics are better.

If you have enough data, for quantitative variables, you may have a look at the relationship shape with splines in a generalized additive model.

You should test interactions that you think may exist and interactions between major covariables that cannot be safely ignored. If you have many variables, you should not test all interactions because there will be too many tests with a high risk of false positive. In my opinion, a significant interaction that is small enough (often on large samples) may be ignored.

Thank you all for your suggestions.

Mehmat Sinan > I have 188 profiles in my data set R2- 35.70%  AIC - 166.34