I have created three logistic models, model 4, 1 and 2, and calculated AICc values for each. Both model 4, with 2 covariates (location and camera), and model 1, with a single covariate (location), have approximately equivalent AICc values (less than 2 points). In this case one should chose the model with the least parameters, this is model 6 with only location included. However, to make things more confusing the likelihood ratio tests for model 4 vs 1 and model 4 vs 2 suggest that having location and camera in the same model is better than just having location or just camera. This contradicts the AICc values. So which model would you choose? I provide an example below. Thanks in advance.
> # location as a covariate on abundance
> m1 m1
Call:
occuRN(formula = ~1 ~ location, data = final)
Abundance:
Estimate SE z P(>|z|)
(Intercept) 2.01 0.704 2.86 4.24e-03
location2 -2.19 0.547 -4.02 5.94e-05
Detection:
Estimate SE z P(>|z|)
-2.32 0.756 -3.07 0.00215
AIC: 162.7214
> # camera as a covariate on detection
> m2 m2
Call:
occuRN(formula = ~camera ~ 1, data = final)
Abundance:
Estimate SE z P(>|z|)
0.682 0.371 1.84 0.0657
Detection:
Estimate SE z P(>|z|)
(Intercept) -2.589 0.763 -3.392 0.000694
camera2 1.007 0.774 1.301 0.193247
camera3 2.007 0.785 2.557 0.010555
camera4 0.639 0.803 0.796 0.425864
AIC: 178.696
# camera as a covariate on detection, location as covariate on abundance
> m4 m4
Call:
occuRN(formula = ~camera ~ location, data = final)
Abundance:
Estimate SE z P(>|z|)
(Intercept) 2.71 0.319 8.49 2.06e-17
location2 -2.25 0.509 -4.41 1.03e-05
Detection:
Estimate SE z P(>|z|)
(Intercept) -4.050 0.616 -6.571 5.00e-11
camera2 1.030 0.620 1.660 9.69e-02
camera3 1.776 0.613 2.897 3.76e-03
camera4 0.592 0.642 0.922 3.57e-01
AIC: 157.2511
> model_list model_names modelsel modelsel
Model selection based on AICc:
K AICc Delta_AICc AICcWt Cum.Wt LL
model4 6 163.25 0.00 0.61 0.61 -72.63
model1 3 164.13 0.88 0.39 1.00 -78.36
modelnull 2 181.06 17.81 0.00 1.00 -88.20
model2 5 182.70 19.44 0.00 1.00 -84.35
Between AIC and the likelihood ratio, the AIC is the more tested and reliable. However model selection should not be based on just a single or two criteria. Other criteria like the BIC (Bayesian Information Criterion), Schwarz Information Criterion (SIC), Hannan \-Quinn criterion, etc. should also be used and a comparison of their verdicts be noted and implemented.
If AICc is your chosen criterion, then model 4 has the lowest AICc value. That's your girl. The likelihood ratio test is supporting that conclusion.
I would go for the model with the lowest AICc but, depending on model objective, ROC analysis statistics should also help deciding.
Try getting the predictive capacity of each model together with AIC or likelihood ratio to decide on which model to select.
firstly how did you determine which variable to include, this might need to be selected in some random fashion way
- Badges
- Science topic