Bivek,

I think you need to be more clear about exactly what your question is. Your statement  about using chi-sq is somewhat vague and is dangerously close to misusing the term 'significance'. what exactly are you asking? It sound to me as though you have a question you need to address, a choice of three techniques and you want to know which technique will resolve the answer to the question best, resolving being your measure of efficacy. If so, then you need to consider relative effect sizes and variances in your three data types, the replication you can achieve with comparable efforts, and then, I'd propose, use power analysis to return the probability of detecting a difference/pattern/trend related to your question using comparable effort levels using the three techniques. It's also possible I've completely misinterpreted your query of course.....  

Bivek, when you compare two methods, you keep one as a gold standard, and compare the other values with the gold standard. You can calculate sensitivity, specificity, positive predictive values and negative predictive values, when it is a 2 X 2 comparison. However, in your case I am not sure what kind of data you have, scale or nominal?

What I see here is, you have three methods. You can compare each with one gold standard. However, if you have categorical variables you can also do Chi Square, but it will be vague to interpret as Andrew has mentioned. I suggest you to consult statistician before you do any method comparison, otherwise the results might be misleading. I hope it was clear to you.

I disagree somewhat with Ashwini, there's no need for a 'gold standard'. If you have one that could allow application of a chi-squared test I guess but efficacy can also be measured as potential to resolve a given effect size at a known type I error rate at a set  effort  level, no standard required just a question of comparative power.

hallo Bivek,   it looks like you're trying to detect a pathogen or something like that ?  So, i'm assuming that your 'efficacy' measure is either numbers of pathogen cells (or similar) or it's a series of "yeses and nos" (that is, the pathogen was detected or it wasn't).  Each of the detection methods is a 'treatment'.

In the first case you have three treatments and a numerical dependent variable "efficacy". The test is therefore a univariate generalized linear model (GLM).  Exactly what form of GLM will depend on the nature of your data, primarily its error distribution. The null hypothesis you test against is that the data from all three tests could have been drawn from a single distribution.

In the second case you have, for each of the 3 treatments, a number of 'yeses' and a number of 'nos'.  The test is therefore a contingency test, such as a chi-squared test. Your null hypothesis is that all 3 treatments should have the same proportions of yeses and nos (the proportion under the null hypothesis being the proportion for all 3 treatments taken together).

Look at C. Dytham's book "Choosing and using Statistics" and follow the decision tree to the most appropriate test for your data.

I suggest to check the probability distribution first (are data normal or not?) and then decide on type of correlation to be used. The distribution dictates whether to rely on Pearson or Spearman correlation, for example...

hey bivek,,,,how r you....???

Thanks Andrew, Ashwini, Andrish, Jiří and Tarushikha for nice comments.

Actually I am comparing these three methods with respect to each other. In one case I made culture as gold standard as Aswini mentioned, and determined sensitivity, specificity, and other parameters of ZN microscopy and Fluorescent microscopy. In this case I used chi.square between ZN value and Culture and similarly between AO value and Culture. There is comparison between two variables.

 Now I am planning to determine case detection rate of these three methods (variables) and to compare these three methods. Is there significant differences among them or not. So to prove there is statistically significant differences in between these three methods, I need some statistical tools. So, please suggest me for that in detail. 

hallo Bivek,

That reads even more as if you're testing for a pathogen, maybe even for TB baccilli in particular.

You mention "case detection rate" - i'm guessing that you mean  the number of cases identified by one of your methods.  Is that correct ?You also write "three methods (variables)" - but it's best to call the three methods 'treatments', the variable is the number of cases identified that you record.

If my guesses about what you're doing are correct then, as i wrote before, you need a contingency test.  (Above I called this the 'second case').

The book I recommended gives you the possibilities and tells you how to carry them out. Checking 'contingency test' in wikipedia would help you too.  And I just googled 'comparing ZN and Fluorescent microscopy'.  I found lots of items on doing this, several with descriptions of the statistical techniques used.

best, Andrew

Bivek,

Your initial chi-sq tests for independence, not difference, important inferential distinction there.

Comparing case detection rates implies something like a comparison of frequencies of positive outcomes  in the three techniques at n different levels within a treatment. Your variable could be detection rate as a frequency (note, not as a proportion) of positives. Suppose you had 3 different detection methods in n replicates of each of 5 dilutions. You could: use a 3 x 5 G test to compare the frequency distributions OR - treat each replicate as an independent sample, with outcome (data) being 0 (no detection) or 1 (detection) and technique and dilution as categorical covariates in a binary logistic model with simple or difference contrasts, or a binomial family GLM. Either of these logistic response models would be more powerful if you have adequate replication (I'd use package lme4 in R), G tests would be sensitive to small sample sizes and low numbers of observed frequencies (n < 5 per category) but you could do the test in a spreadsheet.

Bivek you can use Odds Ratio also. 

Hi Bivek, the Odds Ratio with the 3 treatments (as you have here) is, in effect, a logistic regression ... and a logistic regression is a generalized linear model.But before getting into such details I'd very much like to know much more about your experiments and what you're trying to test..

Thanks Andrish Mhic Da'idhe, Andrew Foggo and Shivani Kachroo for valuable suggestion.

Yes, you are right in my queries Andrish. I am trying to find our the difference in the case detection in these three methods are statistically significant or not. Using Chi-sqaure test, I determined the association between two diagnostic approaches. Now I am trying for the statistical relation among three methods. I am now trying ANOVA test, is this applicable or not in determining there is significant differences in the case detection. Thanks.

Bivek

I feel sensitivity of each diagnostic tool and predictive values may be useful in your case. You may use Cohen's kappa as well.

http://www.ncbi.nlm.nih.gov/pubmed/18224558

http://jmm.sgmjournals.org/content/60/11/1598.full

Approach of likelihood ratio (LR) - LR+ve  and LR-ve can be used tiogether following the table attached for comparing  different diagnostic tests based on qualitative measures  - 

Thank you Ankur Sharma, Miguel Angel Castro Jimenez and Satyanarayana Labani for the valuable suggestion. I made the culture as gold standard and calculate sensitivity, specificity, predictive value positive positive and negative for the other two test methods. The finding of case differences between ZN and Fluorescent microscopy was correlated with Chi square test. Is this fine?

Regards,

Bivek