Hi everyone,
I have an experiment investigating the effect of sediment condition (factor 1 - 2 levels) and the benthic macroinvertebrate community (factor 2 - 2 levels [faunated vs defaunated]) on ecosystem fluxes.
Some animals remained after defaunation, however, so there is a potentially confounding variable in the experiment. Adding a covariate (unremoved biomass of inverts) to PERMANOVA could help to account for this and avoid making a Type 1 error.
There are two opposing thoughts coming out:
1. The covariate should only be added if it has a significant effect
2. The covariate should be added regardless of significance, to account for its effect
Two similar questions online say they should be included regardless of significance:
https://www.researchgate.net/post/How_to_interpret_a_non_significant_confounder_that_kills_my_main_effect
https://stats.stackexchange.com/questions/149384/should-i-keep-or-eliminate-an-insignificant-confounding-variable
So, should the addition of a covariate representing a potentially confounding effect be based on its significance? Your thoughts and reasoning would be greatly appreciated.
All the best,
Sorcha
No. Variable selection should not be based on a p-value. What are you trying to do? Best, D. Booth I am not familiar with that algorithm but are you predicting or explaining? I suggest you look at this link: https://b-ok.cc/book/5243562/0e8c4b
Best, D. Booth
There are several ways to select variables: significance criterion (based on p-values), information criterion, penalized likelihood, change-in-estimate criterion, and background knowledge. As Dr. Booth suggested, variable selection should not be based only on p-values. Check this article for details: Article Variable selection - A review and recommendations for the pr...
Hi David and Belal,
Thanks for responding and supplying information.
David Eugene Booth, you agree with the posted answers in the other questions - interesting. I am trying to assess if the community has a significant effect on sediment-water fluxes (e.g. oxygen) so trying to compare/explain patterns in fluxes among considered factors. The role of the covariate is to account for bugs remaining after defaunation in samples that have 'no community' (i.e. the control).
Md. Belal Hossain, I think this scenario may vary from cases where there are numerous variables and the objective is to reduce the number of variables to obtain a parsimonious model. I cannot recall seeing PERMANOVA (permutational multivariate ANOVA) being used in this modelling context but it could be possible. Their selection criteria could apply here though, which is probably why it has been suggested the covariate should be included only if significant.
I have received another statistician's opinion, which is that there is no definitive answer.
Currently, I am summarising the advantages and disadvantages of its inclusion.
e.g. Advantage: Covariate addition can reduce Type 1 errors.
Disadvantage: Covariate addition leads the order of fit of factors to matter, resulting in multiple tests, which could contribute towards a multiple comparisons issue.
The jury is still out...
Sorcha Cronin O'Reilly The answer is clearly in on the use of p-value selection. It is of no use. See the references in the attached paper theses links: Preprint Boosting and Lassoing Cancer Risk Factors: New Prostate Canc...
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C36&q=automated+variable+selection+methods&oq=automated+var
as well as:https://b-ok.cc/book/5243562/0e8c4b
Best wishes, David Booth
You could just examine level of defaunation across all species
So, if I understand correctly, you have 2 levels of sediment condition and 2 levels of faunation. However you defaunated sediments, some organisms survived and may be having an effect on whatever fluxes you have measured and want to analyse. I wonder if you might be over-intellectualising the problem. From my simple-minded perspective if an analysis based on 'no or low faunal abundances' vs 'normal abundances' shows significant differences then level of faunation matters. If the survival of organisms potentially increases fluxes this would tend to make the test anti-conservative, so type 1 error is unlikely to be a problem. To address your actual question more directly, think what 'significance' actually means in this context. Sure, potentially you could work out some sort of 'abundance/flux' relationship and use it to adjust your fluxes a bit in those samples where you got a worm or two, but would it actually alter your interpretation of the results much?
- Badges
- Science topic
- Similar topics
- Cognitive Science
- Thinking