It's difficult to understand what they mean without any context, but all meta-analysis generally incorporates 'variance estimation'. The tau^2 statistic is an estimation of variance. If you haven't already reported these, then perhaps they are asking you to do so? This is possible for k>1 studies.

Or else perhaps they were referring to robust variance estimation? That is a distribution-free method of meta-regression for dependent effect sizes.

Hello Concetinna,

As long as you have n >= 2 effect sizes, you can compute variance estimates of those ES values (as Jack Henry correctly noted). How stable a variance estimate is does depend in part on how many bits of information comprise it; more is better than fewer.

So, (a) yes, as long as you have two ES values, you can estimate variance; and (b) you can, in an exploratory fashion, address how study variables do or don't relate to differences in ES values (meta-regression, or just subsetting your studies based on those characteristics).

If none of this makes sense, you might consult with someone locally who has more meta analysis expertise.

Good luck with your work.

Hi Concettina Trimboli

The following article can be helpful:

Article Bias caused by sampling error in meta-analysis with small sample sizes

Best regards,

Ebrahim

Though you have not indicated which meta-analytical model you have adopted in your review, it might not be possibly easy to calculate the between-studies variance, tau-squared (τ2), with precision if the number of studies is relatively small (Borenstein et al., 2007). In this case, the fixed effect (FE) model could be your best workable option. The FE model lies on the assumption that all studies are about the same population and have estimates of one true effect size (ES) (Borenstein et al., 2007; Borenstein, 2019). It involves computing one average ES by weighting each ES with the inverse variance method, whereby more accurate study estimates get greater weight. For insightful, relevant guidance, you could refer to Borenstein (2019), Borenstein et al. (2010), Hedges and Olkin (2014), fully cited below.

Borenstein, M. (2019). Common mistakes in meta-analysis and how to avoid them. Biostat, Inc. https://books.google.com/books/about/Common_Mistakes_in_Meta_Analysis.html?id=N4vXyAEACAAJ

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12

Borenstein, M., Hedges, L., & Rothstein, H. (2007). Meta-analysis: Fixed effect vs random effects. https://www.meta-analysis.com/downloads/Meta-analysis%20fixed%20effect%20vs%20random%20effects.pdf

Hedges, L. V., & Olkin, I. (2014). Statistical methods for meta-analysis. Elsevier Science. https://www.elsevier.com/books/statistical-methods-for-meta-analysis/hedges/978-0-08-057065-5

Good luck,

Jack Henry it actually was robust variance estimation. Do you know what package can run this? It looks like CMA can't :/ thanks :)

@Concettina

The package robumeta for R is the only 'out of the box' solution I am aware of. https://cran.r-project.org/web/packages/robumeta/index.html

Jack Henry - a big thanks for your help!!

David Morse Mohialdeen Alotumi Ebrahim Ghaderpour thanks your all your help and suggestions!! A bit more information to contextualise my initial question, it is about robust variance estimation. I conducted a MA where I had multiple outcome measures for the studies but I selected one that 'best reported' what I was most interested in. In my case, psychosocial well-being. The problem is that the studies included in the MA used a range of outcome measures, some measuring strengths and weaknesses of the individual, others measuring post traumatic stress disorder, others, depression, anxiety... therefore I had a real mix mash of outcome measures that I had selected. The response from one of the reviewers was why didn't I use a robust variance estimation. From your experience, can robust variance estimation still be used when the outcome measures are using different variables i.e. the outcomes are quite different, and the sample sizes are really quite small (under 100 in most cases)? I also read something about needing to know the correlation of the outcome measures.

Many thanks again for your help!!