Dear Ankur,

Your question is very important and should be considered in all meta-analysis and systematic reviews. There are many different ways to consider the publication bias and its impact:

1- Graphical depiction of the studies with their effect size in a funnel plot using the "small study effect".

2- Statistical analysis of the asymmetry in the funnel plots using Egger's regression intercept method.

3- Best method is to evaluate the effect of possible publication bias on the results of our meta-analysis using "fail and Safe" methods and "trim and fill" method. These are the methods I personally use in my systematic reviews and are very efficient.

You can find more in the attachment. Several software can also perform publication bias evaluation for you for example CMA. 

Dear Ankur,

Your question is very important and should be considered in all meta-analysis and systematic reviews. There are many different ways to consider the publication bias and its impact:

1- Graphical depiction of the studies with their effect size in a funnel plot using the "small study effect".

2- Statistical analysis of the asymmetry in the funnel plots using Egger's regression intercept method.

3- Best method is to evaluate the effect of possible publication bias on the results of our meta-analysis using "fail and Safe" methods and "trim and fill" method. These are the methods I personally use in my systematic reviews and are very efficient.

You can find more in the attachment. Several software can also perform publication bias evaluation for you for example CMA. 

Thanks Ramin,

The information shared was really useful.

It is bewildering that most of our journals don't want to publish insignificant data. There should at least be a point-counterpoint argument on such studies including statistical analysis that can be included in meta-analysis for future studies so that publication bias can be minimized.

Thanks for sharing the expertise.

Depending on your question you can also use "published/not published" as a moderator to get information regarding what type of of data that you are likely to be missing. Sometimes data unpublished data is not published due to other reasons than "negative" effects. For example, that the data has been gathered with another research focus in mind. 

Apart from that I agree with Ramin's answer, but wanted to add that Egger's is not always a very sensitive measure of publication bias. So, if your sample of effect sizes is small or high in heterogeneity I would interpret an Egger's analysis with some caution. 

Here is a beautiful study that I came across while browsing for literature on this question. I thought it would be good sharing it with you.

Ankur : excellent paper. Many thanks!

The study is indeed excellent! It gave me so much clarity on treatment of all the three types of bias. The language was so simple and explanatory. An ideal study for a journal club!

Basically how can  you reach / get the unpublished data?

Then  perhaps it is a good data that  why  it failed to  be publish

Hi Ishag,

According to Young T et al, 2011; the best way to get unpublished data is to contact the authors through Email which results in greatest response rate, least no. of  attempts and shortest response rate.(Young T, Hopewell S. Methods for obtaining unpublished data. Cochrane Database Syst Rev. 2011 Nov 9;(11):MR000027).

Wolfe et al, 2013 have also described the methods of obtaining unpublished drug trial data. I am attaching the article herewith for your reference.

Thanks Ankur may  be but still  difficult to  obtain

When contacting authors to published articles to ask for more data it may make sense to limit your attempts to recently published articles. I have used a time limit of 10 years back (that is, that the article is published no more than 10 years ago). For older publications the response rate is virtually zero, so it's a lot of work for hardly any pay-off.

So  the time factor is an other factor that  we have to  out in  mins

I feel this is a bit complicated, Ishags earlier comment about quality of data brought me to this question. Even though, I don't consider that unpublished data is essentially a poor quality data, or that all published data is of good quality; however, bias does exist in all data, and this bias is different for different data (also of different types). In case we consider all the studies to have equal weight in in a meta analysis, we would be incorporating another bias in the whole data. My question is : Is there a way of including different data weighted according to bias in a meta analysis so that we can reach a more accurate conclusionÉ

I `m  not sure but  I think  inclusion  criteria  can  grantee this to  some extent 

I think i would re frame it: While conducting a meta analysis, is there a way through which a data can be weighted for bias before including it so that more accurate results can be obtained.

Dear Ankur,

Your question is very important again. However the broad concept of "bias" makes it very complicated.

1- For publication bias, we can not be sure that the bias exists. We can only calculate the effect of "possible" publication bias.

2- For quality of the included studies, what I personally do is sensitivity analysis: I recalculate the pooled summary effect after omission of studies with low quality. However we should define the "low quality" very carefully.

Thanks Ramin,

As always, Great Answer! (Now I would sit and study about it!) I appreciate.

Respects.

I feel that the problem with sensitivity analysis would be that it omits the complete data as low quality, which is not necessary. It may not be completely worthless data. Moreover, there might be some data in the included part that is still erroneous. So isn't that a shortcoming of this process? is it possible that a study wise bias percentage is calculated through MSA or something and a correction be made to each study data based on its error percentage and the corrected data included in meta analysis? 

Dear Ankur,

The statistical method you are looking for, actually does exist. In other words using some weighting factor besides sample size and standard error of each study. This is called bivariate analysis and is routinely used for diagnostic studies to control the "threshold bias". However weighting studies according to quality can be very hard, a number should be assigned to each study according to quality: a process which can be very problematic. 

For example, if we are weighting studies according to quality, how much weight we are going to assign for "proper randomization", "loss to follow up", "blindness', etc, is important. This is not that straight-forward of course.

True! Which means that the best practice as of now for publication bias is 'trim and fill' and for ensuring quality of data is sensitivity analysis. However, there will be shortcomings until statisticians device something new to overcome them. Lets follow the best practice till then. Thanks all, for the enlightening discussion. 

@Ankur sharma. It was a wonderful article. Thank you sir

Thanks

Ankur Sharma for this excellent link