The situation is as follows:
An experiment measured the concentration of a certain chemical in cells at various times after exposure. The results were normalized by the concentration at time zero, so they are fractions. Millions of cells were combined together for the assay. The data points are means and SEMs based on several repeat experiments. I am interested to fit some models to these data and compare them by AIC, but am not sure how to calculate AIC properly because the sample size is unclear. I would appreciate suggestions from anyone interested!
Thank you, Mitchell! Unfortunately I am not certain I will have access to the raw data. Also I am indeed interested in comparing the relative performance of structurally distinct models with different numbers of parameters.
AIC is calculated using the negative log-likelihood and the number of model parameters from a model fit using maximum likelihood estimation.
From the WIKI: AIC = 2k - 2\ln(L)
where k is the number of parameters in the model, and L is the maximized value of the likelihood function for the model.
You should be able to get AIC if you have enough data to fit a model (Mixed model or GEE above). In fact AIC is included in the output for most statistical packages. However, I don't know how you will fit a model without the raw data.
Thank you, everyone, for your interest and contributions! Maybe I am misinterpreting something, but the main difficulty I have is how to compute the log likelihood (and using it, compute AIC), because I am not sure what the sample size is. Is it the number of data points? The number of replicate experiments?
Hi Igor,
as a first step, have you tried, to compare the models using simple CHI-Square (reduced) for each curve and the F- Fisher statistics for evaluating the significance of the difference?
All my best,
Gianni
Hi Igor,
Sample size in the model selection approach is the number of data points (observed values) used to fit and select the competing models. AIC can be computed as (Johnson and Omland 2004):
AIC = 2(n/2ln(RSS/n))+2p
where n is sample size, RSS is residual sum of squares, p is count of free parameters of the model.
Johnson JB, Omland KS (2004) Model selection in ecology and evolution. Trends in Ecology and Evolution 19(2):101-108.
Thank you, Gianni and Luis!
For non-linear models reduced Chi^2 is probably much worse than AIC for comparing models.
Regarding the AIC formula from Johnson et al: I suppose then for the situation I described, n should be the number of repeat experiments times the number of data points in each experiment?
Igor
Hi Igor
as others said the prerequisite for the AIC is that you use the identical (!) data set with same N for all models you want to compare. N = number of observations/data points in your data set you fit your models to. If you have small N use the AICc which has a correction factor for small sample sizes. Most packages will give you the AIC after calculating regression models. Mostly the only thing you have to do by hand is to calculate the correction factor for the AICc which depende an k = # model paramters and N.
see http://en.wikipedia.org/wiki/Akaike_information_criterion, bullet point 3) for the formula of AICc.
- Badges
- Science topic