One of the formulas for Cochran's Q for effect size of path co-efficient is
∑(i=1 to k) w_i *(β-β_avg)^2
Where, wi = sample size of each study.
The Q should follow chi square distribution with k-1 degrees of freedom.
Does the above formula not mean that Q would to a large extent depend on the sample sizes of individual studies? For e.g., if I halve the sample size of all the studies, my Q halves as well. Is that desired behavior and if yes, why?
I am asking because in my study, I am working with total sample sizes (combined for all the studies) of the order of tens of thousands. When I divide the studies in two or three groups (in search of moderator), I invariably get a large value of Q. What does this imply?
looks to me like wi should be the sample size divided by the summ of sample sizes...then Q is a weighted average? Not my expertise, but seems more intuitive.
If you have access to it, see equation 5 in the article linked below. Notice that in the usual approach, the weight is the inverse of the variance (i.e., SE-squared) of the effect size estimate for a study.
So if Sumedha's β_avg is a weighted average of the effect sizes, as in Fleiss's Equation 1, then the only difference from the standard approach is that she is using sample size rather than the inverse of the variance as the weight. And AFAIK, sample size is sometimes used as the weight.
HTH.
http://smm.sagepub.com/content/2/2/121.full.pdf
I have not worked with path coefficients, but I am familiar with Q.
In the definition of Q, w_i is the reciprocal of the (within-study) estimate of the variance of the estimated effect in Study i, and β_avg also uses those weights. Is β_i "normalized" (i.e., scaled) so that its variance is simply the reciprocal of the sample size in Study i? If so, w_i will reduce to that sample size.
It may be counterintuitive that halving all the sample sizes halves the value of Q, if the β_i remain the same. As a measure of the heterogeneity of the β_i, however, Q takes into account their (estimated) variability. For a given set of β_i, the less "noise" each β_i has, the better determined the differences among them are, and hence the greater the heterogeneity. Since you data involve large sample sizes, the heterogeneity is well determined.
The large sample sizes should also allow you to use chi-squared on k-1 degrees of freedom as the null distribution of Q. As I recall, the article by Fleiss that Bruce Weaver cited does not mention (or, at least, does not emphasize) that chi-squared on k-1 degrees of freedom is an approximation for the null distribution of Q when the sample sizes in the individual studies are "large." When those sample sizes are not "large" (the situation in many meta-analyses), chi-squared on k-1 degrees of freedom is not a satisfactory approximation for the null distribution of Q. And, in fact, the actual null distribution varies with the measure of effect (e.g., standardized mean difference, risk difference, log of odds ratio).
As a single number, Q can only do so much to summarize the heterogeneity of your β_i. If you have not examined the β_i graphically, I suggest looking at a "forest plot," which includes a confidence interval around each effect. You did not mention the value of k, but a forest plot should handle a sizable number of studies.
- Badges
- Science topic
- Similar topics
- Social Psychology
- Attitude