I am evaluating an outcome which is an ordinal variable with 5 levels. I am measuring this outcome with a 'gold standard technique' and two other non-invasive techniques in 30 patients. I want to know if the non-invasive techniques are good enough to measure the outcome fairly accurately and which one of the two non-invasive techniques is better? Shall I go for a Kendalls Concordance or an ROC analysis. I am not sure how to compare the two non-invasive techniques statistically. Here is the data file
In looking at your data, it seems that the gold standard is also ordinal. If I understand your question correctly, it seems that you want to know which noninvasive technique is more closely related to your gold standard and all variables are ordinal. If that it the case, then just use Spearman correlations and do a Fisher Z-test of the difference between two correlations.
You could also try doing some inter-test agreement procedure, such as Kappa. You could run both technicques against your gold standard an check which has a better agreement. A good descriptive analysis also would help.
For now I thin weighted kappa is fine. Does someone has some info about the c statistics or the obuchowski measure?
I have looked at your data, and since you don't have that many observations and a rather good agreement, you are likely also to show a rather good overall agreement. However, in this case just making a 5*5 table displaying the agreement will likely give more information, as to how is the disagreement, is there any of the techinqes the over or under estimate compared to GS, is there any difference in the lower or upper classification, or is just random variation.
Kappa is not the correct approach for ordinal data. It is used for nominal data. For the ordinal data like you have, your original hunch of the Kendall's is more appropriate. You might search the literature and see if anyone has proposed a test of the significance of the difference between two Kendalls.
Dennis. weighted kappa is appropriate for ordered categorical data.
http://googleweblight.com/?lite_url=http://www.real-statistics.com/reliability/weighted-cohens-kappa/&ei=LU2ssErV&lc=en-IN&s=1&m=106&ts=1440468165&sig=APONPFmyQMIbi-iIYZ8jEgdItQBMNYnEaA
Yes, it has been used for that purpose. My original statement should have been clarified to mean unweighted kappa. There is literature that is critical of the application of weighted kappa to both the ordinal and other situations and its use at this point must be considered to be controversial. It is also more difficult to explain to reviewers.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics