Really, we always evaluate the model with the available data. For example, when we predict the potential distribution of lesser-known species, we use the presence or absense records available. The empirical data for land use does not the same mean that empirical data for an species (or an invasive species). The requeriment of land use suitability could change across the time and space because of the human factor is not stable in every places. Anyway, this is a good discussion and I am also open to all ideas. 

Hi Oliver, thanks for your response. I'd like to give you some more background to my question to see if you could add anything further to this discussion.

In this case obviously our land use suitability layer has been generated independently of any empirical presence/absence data. The hypothesis is that land use activities (our empirical data) should show a positive, linear correlation with modelled suitability for the activity, and if this is not the case then, all else being equal, land is theoretically not being optimally utilised for that activity.

However, as 100% of the surface (i.e. every pixel) is not being exploited (not necessarily because it is unsuitable), the agreement between modelled suitability and empirical presence/absence can never be 100%, and will in all likelihood be very poor if a simple statistic based on predicted presence/absence vs. empirical presence/absence was reported (due to a large number of false absences in empirical data).

My question is then what statistic would be most appropriate to describe any relationship between predicted and actual presence/absence, given the fact that the empirical data contain a large, unknowable proportion of of false absences? The objective is to produce some quantitative measure of the efficiency of present land use management strategies for a specific land use activity in our project area. Is there any approach you know of from your area of study that might be relevant?

Thanks again!

Is your suitability based on physical variables, like climate, soil, slope? Is "land use" a specific crop?

Hi Hein - the "land use" in this example is in fact ecotourism - suitability, or ecotourism potential, was mapped based on expert knowledge of criteria, including physical variables, that drive tourist decisions and dyamics using spatial multi criteria analysis. By comparing this against empirical data we want to derive a measure of how efficiently tourism is being managed, i.e. is optimum use of resources being made, assuming our suitability layer is an objective representation of suitability/potential. So, this is where I run into the difficulty of describing the relationship between the model and empirical data. 

Craig

If I have understood correctly, your model suggests the optimal areas for ecotourism activities. Your working hypothesis is that the empirical data in the study area should be adjusted to your model for a optimal ecotourism activities. Furthermore, this model suggests new optimal areas . So, the reference image is the output of the model (a raster image), and your comparison image is field data of the study area. It is correct? If that is so, then you can use the same validate-tools for habitat modelling but in the opposite way. 

Although a fish species is used in these papers to illustrate methodology, the model can be modified to assess a variety of watersheds and land uses.

Article A Mid Atlantic brook trout (Salvelinus fontinalis) stream su...

Article A stream suitability index for brook trout (Savelinus fontin...

Hi Craig, you have to evaluate predicted land use suitability versus empirical data, can take the diffrences about de hectares or kilometers and calculate the accuracy in %.

Regards