I am performing a systematic review of prevalence studies. I have pooled prevalence rates using the program MetaXL (double arcsine transformation, random effects model) which gives me a point estimate and confidence intervals for each subgroup. I am trying to figure out how to perform a statistical test for a difference between subgroups.
The problem I am trying to solve is similar to what is presented in this paper:Article Prevalence of post-traumatic stress disorder symptoms in adu...
Subgroup 1 Prevalence: 18.58% [95%CI:12.26-25.8] (pooled 11 studies, n=699)
Subgroup 2 Prevalence: 20.18% [95%CI: 16.64-23.96] (pooled 37 studies, n=6453)
Conclusion from paper is that the difference between subgroups is non-significant (p=0.71), however I can't figure out how they calculated the p-value.
I would also like to be able to perform a test for subgroup differences where there are more than 2 subgroups. For this study, when evaluating more than 2 subgroups, they did not report a p-value.
I have tried creating a 2x2 table and performing a chi-square test but the p-value I get (0.32) is very different.
I want to perform pooled meta analysis. For example depression level in particular population. I have collected Total study sample (N) and depression cases from the 15 different study.
Please suggest how to permored meta analysis.
Mohammad Irshad there are a few different software packages you could use but I used MetaXL which is free and straightfoward to use: https://www.epigear.com/index_files/metaxl.html#:~:text=MetaXL%20is%20an%20add%2Din,to%20do%20network%20meta%2Danalysis
Other key resources I used to learn about meta-analysis of prevalence studies are below:
Article Conducting proportional meta-analysis in different types of ...
Article Methodological guidance for systematic reviews of observatio...
https://jech.bmj.com/content/67/11/974
The p value of 0.71 would have been calculated from a two sample t test of the difference in mean prevalences with degrees of freedom 37 + 11 -2 =46.
you can get the SEs from the 95%CIs
Thanks for the suggestion James Leigh . I would have thought that it is not appropriate to use a two sample t-test considering the data is categorical rather than continuous. Regardless, I was still not able to replicate the p=0.71. This was my working:
Sample 1: Mean=0.1858, SE= (0.2580-0.1226)/3.92=0.0345, n=11
Sample 2: Mean=0.2018, SE=(0.2396-0.1664)/3.92=0.0187, n=37
Gives: t=2.017, df=46, p=0.049
This may be because I did not/don't know how to account for the following:
-The pooled prevalence was calculated using a random effects model so each prevalence is weighted differently (unlike a typical mean)
- The 95% confidence interval is not symmetrical (the point estimate does not lie in the midpoint) so I'm not sure if there is a different way to calculate a standard error
If you have been able to replicate the p-value, would you mind sharing the calculation?
The paper describes use of a double arcsin transformation . I'm not sure how they did significance test.
Using a t test would give SE of difference:
sqrt ( 0.0345 x 0.0345 + 0.0187 x 0.0187) = sqrt 0.00154 = 0.0392
and t46 = (0,2018 - 0.1858) / 0.0392 = 0.016/0.0392 = 0.408 ;P about 0.68 , which is pretty close to 0.71.
That makes sense, thank you. I realised that I accidentally plugged the SE into an online calculator which required a SD without converting.
Would you suggest a t-test as the best option for examining a subgroup difference (and ANOVA for comparing multiple subgroups)? Or do you think it is better to use a chi-square test using only the prevalence point estimate and n=699/n=6453 (ignoring the confidence interval and number of studies)?
I would suggest t test and ANOVA. The confidence intervals effectively variance weight the studies, while a total pooling effectively weights only for study size.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics