Hi,
I built a Maxent model using 15 replicates to see how much variation there is using the same parameters. My problem is that the different replicates present different Analysis of variable contributions. Per example, in model no 1, croplands is the most important variable but in model no.2, it is altitude.
I am wondering if I should present the results only for the model with the best AUC (and ignore the fact that other good models sometimes predict a different order of importance for the variable), or if I should use the model showing the mean results of the 15 models created (replicates)?
Thank you.
Dear Marie
I have seen both approaches you refer to used in papers, however, in my personal experience is best to use a replicated method.
Eg. The number of replicates of the model is equal to the number of sample points. So the average model, in the end, was both trained and evaluated using all points.
See the "Calibrating MaxEnt" section of
Article Applying species distribution modelling to improving conserv...
Dear Marie-Helene,
Although it is a classical one, this paper may have relevancy to your question:
Article Ensemble forecasting of species distributions
HTH,
Ákos
Please refer to our bear article (keyword: Majella) for a similar finding and our explanation.
Based on your short statement, it seems cropland and elevation are correlated environmental predictor variables. Or in the words of the referred article, both contain the same spatial information. Evidently, such correlations are common in high mountains.
From a broader perspective, we may realize that there is never a single superior model. It depends on the adopted selection criteria based on the research objective. In my experience, we can built a number of models with practically the same high AUC from the same data set. By substitution of one predictor variable by another, we increase our understanding of the environment of the species. Beyond AUC (or AIC), parsimony in the number of variables necessary to reach an acceptable AUC may be a selection criteria of the "best" model.
When we use a broad set of environmental variables (meteorologic, bio-climatic, land use, soil, elevation, vegetation etc.), we may remove highly correlated variables before modelling to avoid multicollinearity and improve transferability of maxent models (e.g. see our recent ASF and dehesa [Duque-Lazo et al.] models). Alternatively, we apply stepwise backward removal of predictor variables as in our bear article for the same reason and learn in the process about the environment of the species. Removal per category of variable (e.g. bio-climatic first) showed to be enlightening as well.
enjoy
Dear Marie-Helene Hachey
First, you have to investigate the collinearity and correlation between the used variables. After that, the best way is the simply calculate the mean or the median across your models (replication).
Thank you for your answers! I already tested my data for correlation and the maximum value of "r" is 0.4, which is good. I will use replicates. But there is still something I am not sure of: in my results should I discuss the results of the "mean model" (e. g. use the graphs showing the mean values (see file "hiboux_roc")) or should I present results for only one of the individual models (see file "hiboux_14")? In other words, in litterature using Maxent, do authors present results for "mean models" or only "single models"?
Dear Marie,
obviously you should evaluate the model you end up using, i.e in your case apparently the averaged model. Mind though that thresholding individual models and then integrating them, as done by Araujo & New 2007, may be more informative.
Additionally, in both cases you may want to provide both auc of mean model and the mean auc and sd of auc of all replicate models.
Dear Marie-Helene Hachey
The best way is to represent and evaluate the mean model, in your case "hiboux_roc". Apparently your mean model is robust (AUC= 0.948; 0.5 (random)
With reference to our bear and endemics articles, you may consider to enter the various categories of environmental predictor variables stepwise, let say elevation alone first or at least without land use/cover ("cropland"). You may generate a better understanding of the environment.
Further, I wonder whether you have considered to apply a method to minimise multicolinearity of the environmental variables by posterior stepwise backward elimination as in the above papers or a priori by PCA (our recent ASF article)
I think the question asked by Marie-Helene Hachey is whether to use the mean ROC curve generated from replicates or without replicates. I am also facing same problem. If I go by the available literature, the ROC curve gives training data AUC, Test data AUC and Random prediction AUC that needs to be interpreted but when when model is run with replicates the the ROC curve gives Mean AUC, Mean +/- one stddev and Random prediction. So, which one of the above is to be interpreted in final synthesis. Kindly share your opinion especially Hein Van Gils Sir
- Badges
- Science topic
- Similar topics
- Geoscience
- Atmospheric Sciences
- Climate Science
- Climate Change