I am analysing complex survey data in stata. Tests like lrtest, AIC or BIC are not supported by svy and thus I could not use any of these to compare successive models for improvment. I have read about using the wald test. The issue is that the wald test would test paramaters, not the model as a whole. Is something available to test the overall model and so be able to compare successive models? I would really appreciate if someone out there could suggest a way out...
Kindly
Dear Tieba,
Not long ago I had to analyze DHS dataset and, actually, yes, you can only use the Wald test when performing regression analysis. You're right that the Wald test does not assess the overall the model as a whole like the LRT, but it has 2 advantages:
1) there is nothing you have to do to perform the Wald test; once you run the svy logistic outcome variable etc. the p-value you get is already the one from the Wald test, contrary to the LRT in which you have to nest models with and without a variable
2) the Wald test will give you a p-value per each level of an exposure if such variable has more than 2 levels, which the LRT does not. For example, imagine that in your analysis you have grouped age into 4 age groups. The youngest group is coded as "0" and is your reference. You want to assess whether age is associated with outcome "X", therefore you do your model (e.g. svy logistic: outcome"X" i.age) and you'll maybe see that compared to the younger group, only the oldest will have an association with the outcome, while the intermediary group may not have.
You'll not get this with the LRT which will only tell you if one variable is overall associated with the outcome.
It's true that the Wald test has also some limitations; for example, if you want to see whether inserting a variable in a model with or without the linear trend, with the LRT is easy, while with the Wald test you have to check the results visually and decide whether the linear trend is better than showing the results not in linear trend. Also checking for interaction with the Wald test is possible, but more complicated than with the LRT
Unfortunately, when analyzing survey data, you don't have many options... so to my knowledge, the Wald test is the only one...
If you want to see the paper I published when analyzing DHS data and the methods I used, see: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0132357
I hope this helps
Cheers
Rodolfo
Hi Tieba,
instead of using the svy prefix you can specify your models using the weight and robust options. This will yield exactly the same results. Doing so, you can use the estat ic command after running your model to obtain BIC and AIC values.
Example:
logit y a b c [pweicht=weight], robust
estat ic
Best,
Alex
Thank you all for commenting. Indeed, I am testing logit models. Rodolfo and Alex, your comments are both very useful. Putting all together, I should be able to find a way forward . Thanks again.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey