Hello everyone. I am a Master's student currently carrying out a systematic review for a research project. My question is: does cold-water immersion or swimming benefit the human immune system?
I have identified 22 studies which meet my search criteria and I am in the process of assessing them. I would like to carry out a meta-analysis but I have not done one before and I'm not sure if it will be possible.
The studies I have might just be too diverse in terms of the types of cold water exposure they assess. I have read that a meta-analysis is not desirable if the study methodologies are too different, as it will be meaningless or misleading to combine the findings of studies carried out in a very different way.
However, I can't find any guidance on how exactly to assess if studies are suitable for meta-analysis. Most of the online guides I've read (eg Cochrane handbook) sort of skim over this section, and give the impression that it's really down to reviewer discretion? This seems a little rough to me, and I feel that there might be some kind of statistical test out there which I can use instead? The online resources provide plenty of information about assessing study heterogeneity after the meta-analysis has been carried out, using I2 or Chi2 tests or similar, to assess whether the heterogeneity of results are compatible with chance or not, but, unless I'm missing something, I can't find any tool for assessing heterogeneity of methods.
The studies I've found have considerable variety of exposures:
Static immersion, swimming or other exercise (cycle ergometer etc) while immersed
Water temperatures ranging from 0-18°C
Duration of immersion from 20 seconds to 3 hours
Length of study from a one-off immersion to an entire five month swimming season
Some studies also include adjuncts like saunas, cold-air cryotherapy and pre-exposure warmup exercise
Participants vary quite a bit (by winter swimming experience, age, gender etc), but not as much as exposures.
Outcomes are mostly fairly similar (white blood cell counts, immunoglobulin, cytokines etc)
Another systematic review I've found on the effects of exercise on the immune system (Chastin et al., 2021, https://doi.org/10.1007/s40279-021-01466-1) does carry out a meta-analysis on what seemed to me to be studies with quite diverse interventions. For examples a study which used a cycle ergometer and one which used running, walking, climbing and gymnastics both went into the same meta-analysis because they had similar participants and duration of study. But I can't anything in Chastin et al.'s methods to say how they decided which of their included studies were suitable for meta-analysis.
Any advice that anyone can offer is great. Bear in mind that I have never carried out a meta-analysis before, I don't have much experience in statistics and or come from a maths background, so please keep any explanations as simple as possible. Thanks in advance, Josh
First and foremost, your research question and study eligibility criteria have to be justifiable to an expert in your research field. Remember that because an expert or two will review the product (e.g., manuscript, conference presentation) of your systematic review/meta-analysis. There is nothing wrong with having heterogeneity due to variability in study methodology, but you do not want an excessive amount of variability because you will not have the statistical power to attribute heterogeneity in study effect size to the variance in study methodology. However, with 22 or more studies, you should have decent statistical power when applying subgroup analyses and/or meta-regressions in an attempt to explain heterogeneity. There is no magic rule that can tell you how much variability in study methodology your systematic review/meta-analysis can "tolerate." You learn that with practice and sound guidance. I recommend that you seek out an expert on your research question and one on conducting systematic reviews/meta-analyses. Do not try it all by yourself the first time around. You might learn more, but it will be more painful.
As a casual comment, I highly recommend that you format your research question using PICO format if at all possible. The "benefit the human immune system" phrase caught my eye. A lower number of leukocytes in the blood and/or a tissue is not necessarily beneficial if that is what you were thinking. Be very specific when you define "benefit the human immune system." A good advisor will help here.
These guidelines here might be helpful for you: Space Biomedicine SR Methods (google.com)
Hi guys, thanks for your replies.
I have now realised that there can't really be a statistical test for heterogeneity of methods because they are not numerical values like results are. So it really does just seem to be down to my discretion. I'm now leaning away from meta-analysis because of the diversity of methods in the studies I've found might make it misleading to pool them. At the very least I'd have to divide them by different patient populations, which would reduce the statistical power of the analysis. I think I will focus on a narrative synthesis and study quality appraisal for now, but may consider a meta-analysis later if I think it would be useful.
Gordon - thanks for the tip about PICO questions. I might change it to something like 'what effect does cold-water swimming/immersion have on the human immune system', and then discuss if the reported effects are beneficial or not in the discussion.
Tobias - thanks for the guidelines link, I will take a look
Josh
http://www.jennifervonk.com/uploads/7/7/3/2/7732985/meta_analysis_fixed_vs_random_effects.pdf
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis