Glass stated that 70% of the problems are solved with secondary data. Data secondary analysis is the re-ana-lysis of data for the purpose of answering the original research question with better statistical techniques, or answering new questions with old data. Some of our best methodologists have pursued secondary analyses in such grand style that its importance has eclipsed that of the primary analysis. For example, Glass suggested that secondary data cannot be inferred by looking at or reviewing it and need to be interpreted. In result, Glass’ (1976) meta-analytic techniques Was raised. Actually, the meta-analysis found little relationship between study findings and design features of the experiments, settings for the studies, or manner and date of publication of the findings. And the effect size was introduced as a meta-analytic beating heart.
Methodology for conducting a systematic review
and performing meta-analysis assumes the maximum objectification of the combined and analyzed data. However, a number of points reveal weaknesses and complexities of these methods.
First, the question that is being answered through systematic review and meta-analysis should be formulated as clearly and concisely as possible. The same question should become a criterion for including studies in the analysis, although there may be many initially asked questions in the selected studies, but for each of these questions a separate meta-analysis should be performed. The best option involves qualitative studies with the same null hypothesis, but in practice it
is not always found.
Close to this is the problem of data multiplicity: it is common for individual studies to find the influence of the independent variable (factor) on several dependent variables or on several aspects of the dependent variable. It should be borne in mind that the tempting decision to combine such results from different studies will undoubtedly increase the power of meta-analysis, but it is fraught with a high probability of erroneous, unreliable, and
sometimes paradoxical conclusions due to conceptual confusion: after all, not all data included in the analysis will be independent. Therefore, it is preferable in separate meta-analyzes to study the differential effect of the independent factor on the dependent variable (or a homogeneous group of dependent variables).
Secondly, data search should be as complete as possible, which is very difficult. First of all, this is due to the fact that, as a rule, only positive results are subject to printing, and negative data are not published - the “file drawer problem”, fraught with a systematic publication error, bias of the result towards a positive effect ( "Publication bias"). Therefore, when searching, it is very important to research not only the available
databases, but also abstracts of unpublished articles,
"Gray literature": conference abstracts, reports, unreferenced and unreferenced journals, data
pharmaceuticals, etc. Of course, in a perfect meta-analysis
it would be necessary to carry out statistical processing of the primary data of patients (included in the selected studies) through a request, correspondence with the authors of the studies, but as everything is ideal, such an analysis is practically unattainable, although banks of primary data of patients on some medical problems are being created abroad. Typically, a meta-analysis includes the combined and already processed data of each study, the results of the primary statistical
analysis. Unfortunately, the bias of journal editors and the developed complex of authors about negative results, which are most often not published, leads to bias (bias) of results, systematic error, and reduces the reliability of conclusions as a result of such a meta-analysis.
At the same time, the so-called the effect of the "curse of the winner" ("winner's curse" - auction, financial term), because usually the first to publish is the author who has received the largest and most significant positive effect. The rest of the authors who conducted research earlier, but did not receive a significantly pronounced effect, do not publish their results. Therefore, it is not uncommon for a situation when it is the first publication that causes heterogeneity (heterogeneity) of data: in other works, as a rule, there is also an effect, but much less.
In addition to the problem of the "first publication", there are other stumbling blocks: according to the methodology, only qualitative studies should be selected for meta-analysis: works without randomization, with incorrectly formulated tasks, with incorrectly performed statistical analysis, should be eliminated, because, for example , nonrandomized trials tend to overestimate the effect of treatment and distort the objective assessment of the event. Ideally, homogeneous studies are included in the analysis: from a meta-analysis perspective, homogeneity is the combination of the results of one trial with the results of other studies. Assessment of research heterogeneity is carried out from the standpoint of common sense ("clinical heterogeneity" is determined) and with the help of special statistical methods (statistical heterogeneity).
Basic steps and rules for meta-analysis step by step set out on the pages of the Cochrane Collaboration and in a special instruction on "Preferred Reporting Parameters for Systematic Reviews and Meta-analyzes" (PRISMA) or the QUORUM requirements (on improving the quality of meta-analyzes of randomized controlled trials).
Regards, Pushkin Sergey Viktorovich
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Materials Science
- Materials
- Inorganic Materials
- Ceramics
- Glass