Hi everyone,
I'm trying to determine which test to run to assess the accuracy of a model that's classifying vegetation. I have ground-truth values and the values that the model has produced. I've considered Pearson Correlation and Intra-Class Correlation, though there are many tests, so I'm stumped on which to decide on. I've seen past literature using Pearson Correlation though my data aren't normal even with a log transformation.
Thanks much!
Hello Charles,
Can you please explain/think of your objective a bit more clearly? It is not clear whether you want to assess the accuracy of the classification method or you want to run some goodness of fit tests to understand if the model lacks something. If it is the former (as it seems from the way you framed the question), a confusion matrix might be most suitable to understand the results of your algorithm. There are several other measures to give an idea about the performance of a clustering/classification algorithm, Cohen's kappa, purity index, RAND index being some examples. Each one has its own advantage and disadvantage. I would strongly recommend you to read about them and make a decision depending on the type of dataset and problem you're working on. Hope that helps!
Thanks.
Hello Charles, since you got the ground truth you really can check how well the prediction model performs. Like studies with AMOS/LISREL/PathAnalysis the best way to show the validity is to show more than one index, show a number of different performance measures.
Not only percent correct (with a fixed cut off score), not only (non parametric correlation). A score showing combined detection and false alarm rates is the AUC value. This is a good start. both SPSS as R can give you the AUC measure. SPSS does give the AUC value when you perform an ROC curve analysis. Perhaps the most important validation comes from clinical users, giving just their opinion what is best for them.
Eddy
Soudeep Deb and Eddy Brand
Thanks much for your answers, I really appreciate it!
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling