Hi Luís,

The approach to evaluate the accuracy of the maps depends on the purpose of the map.

In general, you can directly compare the index value for the different mappings. The bigger the kappa the better.

However, it may be even more useful to analyze the confusion matrices.

The following paper is one of the classic references on accuracy assessment:

Article A Review of Assessing the Accuracy of Classifications of Rem...

Best,

The goal of Kappa coefficient is to measure the agreement between two systems. In remote sensing, it is applied to define the classification precision of algorithm comparing the output of algorithm with already classified image/dataset (waiting result).

If you have list of algorithms and their Kappa, the strongest Kappa identifies the most precise algorithm, however, this case is true, if the classified categories are similar in all cases. Kappa considers all categories as equal sets (normalizes amount of samples), therefore it is more preferred then total accuracy, which increases precision when there are more samples of class with better classification precision neither with worse classification.

(If one class isn't recognized very well, Kappa is small despite of high precision of other classes)

Kappa coefficient and total accuracy are calculated from confusion matrix, therefore good style is to provide confusion matrix, which can be applied to calculate different coefficients for comparison.

Therefore, if confusion matrices are provided, the more correct solution is to rewrite confusion matrices only with common classes and recalculate their Kappa coefficients and compare them.

dear luis

please have a look:

Article Comparing global vegetation maps with the Kappa statistic

https://www.tandfonline.com/doi/abs/10.1080/02693799608902099

http://www.css.cornell.edu/faculty/dgr2/teach/R/R_ac.pdf

https://www.academia.edu/2861251/Assessment_of_map_similarity_of_categorical_maps_using_Kappa_statistics

http://web.pdx.edu/~nauna/resources/9-accuracyassessment.pdf

you can select suitable method among them.

Thanks!

Hi Luís,

perhaps you should skim these papers

Article What is the difference between two maps? A remote senser’s view

Article Classification accuracy comparison: Hypothesis tests and the...

Also, kappa has been criticized, so consider if kappa suits your problem.

Article Harshness in image classification accuracy assessment

Article Death to Kappa: Birth of quantity disagreement and allocatio...

Cheers

Hugo

Hi,

you can check the personal web-page of Robert Pontius: http://www2.clarku.edu/~rpontius/

All the best

Nuno

Dear after you do accuracy assessment, then you can do statistical analysis to do your comparison.

My regards