I test 2 different methods (A and B) on the count of specific bone microscopic structure. My goal is to assess that method B is reliable in the count of those microstructure. So I count the number for each of those methods in the same specific area of the bone for 30 individuals. With method A (which is the gold standard method), I counted in total for the 30 individuals 193 structures and with method B, I counted 184 structures. However, I have only 127 structures in common for both method. What statistic can I use in order to test if it's significant? Does khi carré works even if the total number differs (e.g; method A:193; method B: 184)?
A is your gold standard, that means you assume the resutlts from A are correct.
You thus in B have k=127 correct results of n=184 counted structures in total.
Give an error-rate r, the number of correct results in such experiments is binomial (k,n,r)-distributed. With this distribution you can estimate r. The likelihood will have the shape of a beta-distribution, the maximum likelihoos is at k/n, and you can get the confidence interbal from a beta-distribution.
If you have a statistical software you can use a generalized linear model of the binomial family to get the confidence interval ("intercept-only model", as you don't have any predictor).
For any r outside of the (1-alpha)-confidence interval the p-value is smaller than alpha. You may then judge if you consider the rate of errors made with method B is relevant.
Hi, for bone you can use Gordner trichrome staining on bone sections obtain after embedding in methyl metacrylate and cut with a microtome. After you make bone histomorphometric measurements with an automatic analyzer and a specific bone software for counting, etc..
Ciao, if i am to assume that secondary osteons are the micro-structures counted, or maybe you counted fragmentary secondary osteons (which are more difficult to count) .....a mean ?? of 193 verses 184 is not that significant (specifically, if you are counting osteons to be used in an age predicting equation) . if I am right in my assumption....One thing I would do first before comparing methods is to see if you come up with 193 or close to that count using the gold standard. Only then will you know that you are internally consistent (intra-observer error) with the gold standard before comparing your counts with the method B. Maybe you have already done this..if so great...
let me know if i am on target with your question, if not please provide more details to your variables.
Not sure of your issue. Are you testing the reliability in terms of the actual osteon counts between the two methods, or is it the results for predictions, e.g., age estimate by the two methods. If it is the first, then it would seem that the differences in counts are a matter of repeatability, unless the the variable definitions differ between the methods. If it is the second, then it is a matter of predicting equation performance. Not sure how one method is the gold standard. Would not the actual (known age) be the gold standard? Are you sure that you are not interested in accuracy? If reliability really is the question, then wouldn't it be a matter of which method produces consistent results, not necessarily more accurate. Maybe I don't understand the question.
Hello Sam and Robert, actually I finally found the answer with a simple chi squared according to my data. I wanted to see if polarized light microscopy was comparable to SEM in terms of zonal osteons. I wanted to applied the same methodology than in medicine for method-comparison with a Bland-Altman test but it does not work with the data I'm using here so as you said Sam, is more about consistency between the methods than reliability or accuracy! Thank you all for your answer!
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Materials Science
- Materials
- Inorganic Materials
- Metals
- Alloys
- Steel