I am interested in comparing model fit among proc mixed repeated measures models for 8 different outcomes. Fit Statistics, AIC, AICC, etc., were used to select the best fitting model for each of the 8 outcomes, however, I would like to be able to compare the best fit model for each outcome to each other. For example, if I wanted to compare how much variation in weight was accounted for by the best fit model for weight to how much variation in height was accounted for by the best fit model for height, what metric would I use? I have found some reports and publications that use Covariance Parameter Estimates to compute the Intraclass Correlation Coefficient(ICC) to compare models employing different hieratical structures for the same outcome and I wondered if the ICC could be used for my situation as well.
Joseph I did not understand your motivation about comparing accounted variation between different variables. Could you explain your aim more why you want to do so?
Thanks for the response Mehmet. For example, let’s say height and weight are metrics for measuring the effect of a new food additive on growth in foals (young horses) and that resources are limited such that having only one outcome for growth over time would save time and money. Both candidate outcomes are modeled over time (repeated measures) with the independent variable (food additive) and a set of covariates that are known to affect growth. The best fit model is found for each outcome, but which of the two outcomes can be modeled more accurately (more variation is accounted for)? If one outcome is better fitted or predictive with available data then it may be worthwhile to focus on that outcome in further analyses and future research.
OK I understand better now. Actually I misread and thought that there were 8 independent variables and that confused me. But I see now that there are 8 possible outcomes and we are selecting the best-fit model.
ICC, AIC and AICC etc will not help in this case. ICC is usable if you have a cluster or group. You need a statistic like an R-squared to measure variance explained by explanatory variables. I have attached references about this. I understand that this is an open research subject now.
IMO Selecting best model can also be done by analyzing residuals obtained from different models.You could check to see if they are normal, homoscedastic, small and the model which gives you best residuals might be considered best.
http://stats.stackexchange.com/questions/7240/proportion-of-explained-variance-in-a-mixed-effects-model
http://thread.gmane.org/gmane.comp.lang.r.lme4.devel/3281
- Badges
- Science topic
- Similar topics
- Books