To evaluate interaction is one of the most important objectives of fatorial experiments. The use of less rigorous level of significance for interpretation of the interaction, maybe it can catch important effect. In some situations, even without significant interaction (p≤0.05), the decomposed interaction demonstrates the differential effects of levels of factor B for each level of factor A or vice versa.
Could I use p≤0.15 for the interpretation of the effect of the interaction, keeping the usual p≤0.05 for the isolated effect of factors and comparisons? Is it adequate for analyzing agricultural field experiments?
Do you have current article recommendations on this topic?
Thanks!
The level of significance is your choice. You decide based on scientific and data assumptions. Analyze collected data as data. The data show only data results in the context of data assumptions and conditions prior to collection. Make scientific interpretations after all data conclusions are made.
It's interesting than you seem to consider 0.05 a "rigorous level". It is more likely to get p
You say:
"even without significant interaction (p≤0.05), the decomposed interaction demonstrates the differential effects of levels of factor B for each level of factor A or vice versa."
Can you give more details about how this is demonstrated? Significance testing is one approach to this (and as Jochen Wilhelm says and it is argued more in 10.1038/s41562-017-0189-z the .05 isn't particularly rigorous), but other approaches (e.g., Bayesian approaches) are available. It is worth you saying, also, if you have already looked at your data and have found a p value for the interaction, or this is part of your analysis plan for an ethics proposal or pre-registration.
This question came up in a conversation with a friend. Where we commented on the subtle differences, which are difficult to be captured in agricultural field experiments. Sometimes it can catch important effect for farmers. Example 5-6% increase in soybean grain yield, which is significant by the Tukey’s test (p ≤ 0.05), but with non-significant interaction effect by ANOVA, F test (p ≤ 0.05).
This friend of mine mentioned the article by Perecin and Cargnelutti Filho (2008), see: http://dx.doi.org/10.1590/S1413-70542008000100010 , where the authors recommend the use of p = 0.25 for the interpretation of the effect of the interaction "for experiment" remaining usual p = 0.05 for effect "for comparisons".
Maybe I explained myself wrong. Not that p=0.05 is rigorous, but p=0.15 is less rigorous than p=0.05. Such a shift in significance levels will reduce considerably the type II error rates at the expense of somewhat increased risks of type I. Correct? but what is the cost benefit of this?
I understand that if the experimenter wishes to reduce the occurrences of all types of errors, he must improve the precision of his experiment. What can be done by increasing the number of replications for example, which is sometimes unfeasible. But I would like to know if in some cases changing the p-value, procedure can result in a less costly balance of statistical errors.
Although I'm a bit skeptical about it. I would like to know if this can be a valid tool in data analysis from agricultural field experiments.
Thank you for all your valuable contributions.
- Badges
- Science topic