I have two models having the same fixed effects - one assumes the correlations of the repeated measures is autoregressive corAR1 while the assumes the correlation structure is independent. How can I compare the two models?
This is a good question. There's been some work by Pan that proposes the use of quasi-AIC (QIC) for comparing GEE models...here's some R code for applying Pan's QIC:
I may be off base because I don't use GEE much, but don't you have a deviance statistic (e.g., log likelihood)? These models should be parameter-nested, so you could use a likelihood ratio test based on the difference in number of parameters estimated.
This is a good question. There's been some work by Pan that proposes the use of quasi-AIC (QIC) for comparing GEE models...here's some R code for applying Pan's QIC:
@J. Patrick Kelley: Thank you for the link.its been very helpful. I have managed to get CIC values for both models, unfortunately QIC values only retained NaNs but based on CIC I've opted for the independent correlation structure.
@Patrick Malone: To the best of my knowledge GEE do not use full likelihood estimates hence my predicament.
Raymond, if you're not changing the fixed effects, it should be ok with restricted ML (at least that's the case in mixed models). As I said, though, not a lot of experience.
Patrick, actually am more comfortable with REML myself, i made the shift to GEE because in mixed models; I've specified the correlation as 'AR1' and from the output I can get an AIC, however, when I specify the correlation structure as 'independent' the AIC is not obtainable (i dont know how to get it) thus i can not compare the two models. In GEE library(geepack) am able to get a CIC value for both models which is a bit helpful. I'd be happy finding a way of comparing the two models within library(nlme) without leaping into gee.
I'm not sure about CIC at all. From what I've seen, QIC is now a loose standard for model selection in GEE. I take it that you applied Pan's R code for calculation of QIC. If you got NaN values returned, then maybe there's still hope. It's likely that there's something about the original data that is causing some issue (like presence of NAs or something odd like that).
I can confirm I do not have NAs, my outcome variable is continous..other than that i cannot think of anything that would be an issue. However, i think the problem lies with the way i applied Pan's R code because the warning messages are all based on the object 'mu.R' that is what's producing Nas see (below). Now since mu.R is based on fitted values of the geeglm i dont understand why i would be getting Nas because I do not have a zero for any of my fitted values??
I have an additional question: What formal test can we use to compare two GEE models? Can we use Wald test or some kind quasi-likelihood ratio test to do model selection? My exam question is asking to construct a test of two nested GEE models. Not sure how to do it.
Granted my answer is a year-and-a-half after the initial discussion, but I have been exploring this topic myself lately. GEE is a quasi-likelihood method, thus the likelihood-ratio test is usually inappropriate as the estimating equations are not maximum-likelihood. Pan (2001) proposed an analog to the AIC called QIC (Quasi-likelihood under the independence model criterion). But this criterion does not always perform well. Hin and Wang (2009) have proposed a better performing criterion called CIC (correlation information criterion). But not all software has incorporated this yet.
Here is a link to a SAS macro to calculate both these criteria: