It seems that your expriments are actually ment to be "hypothesis generating", not "hypothesis testing". In such a case, you should not interpret the absolue value of the p-values. You may use them as some kind of "signal-to-noise ratio" that may be used to rank genes or associations (according to their signal-to-noise ratios). There is no (sensible) significance or confidence that would apply for any kind of "conclusion" you can draw from such data. Formulate your hypotheses and then design effective experiments addressing these hypotheses. The data analasis of these data may then have well-defined statistical hypotheses to be tested, what will will give you the information if the sample size was large engough to be able to interpret the observed statistic (e.g. a parameter or coefficient in a model) w.r.t. the value of the null hypothesis being tested. If this is "marginal", you should have used a larger sample size.

Jochen Wilhelm Thank you for your answer!

Charalabos Antonatos ,

I don't subscribe fully to your statement. The meaning of calculating a p-value (or doing a significance test) is to find out if the data provides sufficient information for a confident interpretation of the estimated statistic (e.g. a mean difference, or a slope) relative to some fixed value (the "null hypothesis" H0). Often this fixed value is zero (zero mean difference, or zero slope), and rejecting this hypothesis means that we can interpret the sign of the estimate (is it positive or is it negative). This is the minimum requirement to make any interpretation at all. It is useless to interpret the actual value of the estimate if it is not even (sufficiently) clear from the data if we can trust at least the sign of this estimate.

The significance test is a crude "first line of defence" against making interpretations (relative to H0) that may not be sufficiently well supported by the data.

If you can reject the "zero" null hypothesis, you can at least interpret the direction, but there is still no information given if the estimate is ok or very misleading. This information can be obtained indirectly by testing the entire spectrum of possible hypotheses (not only zero). The interval of hypotheses that can not be rejected (at alpha) is clled the (1-alpha) confidence interval. It gives you a range of values that are "not too incompatible" with your data. If for a given H0 p0.05, the CI will contain H0.

If p>0.05, the CI can be useful to see if the data is sufficiently incompatible with a "relevant effect". So one might conclude that the effect is not relevant.

One may use the CI to give a range of values that are not too incompatible with the data as an interval estimate of the effect size. It is important to interpret the entire interval. If it would be important that the effect is either positive or negative, an interval containing zero is not sufficiently informative, and this is checked already when one sees that p>0.05 for the "zero" H0. If it was important to show the the (possible) effect is negilgible, it would be more direct to test the "non-zero" H0 (value fixed at the minimum relevant effect). If this can be rejected and the estimate below that limit, the conclusion is supported by the data.

If you really go for estimation, significance testing is not useful at all, and it is irrelevant wheter or not you can reject some H0. The question is directly: "what is the value, and how precisely can I estimate it?". This is a very difficult task. Simply using the CI for this purpose can be tricky, because in any concrete case the estimate of the variance may be grossly bad or wrong, giving a very wrong impression of the precision. This depends a bit on the sample size; it becomes less likely to be really wrong with increasing sample size. So any reasonable attempt to etimate the value of an effect size will usually require much larger sample sizes than just significance testing. And then there is a further logical challenge: Both, MLEs (or LSEs, as a special case) and CIs are actually statements about the data, given hypotheses about the effects. If you wnt to have a statement about the effects, given the data, then you need Bayesian statistics. Again, for large(!) samples Bayesian credible intervals and ferquentist CIs iwill numerically converge, but their theoretical justification and interpretation is different. For not-so-large samples, the chosen Prior will (and should!) have a considerable impact on the result, and chosing a Prior requires a lot of understanding.

I agree with Jochen. Based on reading the title and the last message containing the word "marginal p-value" I needed to think of this blog: https://mchankins.wordpress.com/2013/04/21/still-not-significant-2/. Suggesting that either display your p-value without judging it based on a stark cut-off an look to it in the context of your study (without making any claims to it if certain premises have not met) or suggest it is significant/non-significant. It is a bit weird to suggest that it is marginal significant it is either yes/no (bit weird though).

Yes, that's what I wanted to say: the whole purpose of a significance test is to see if we can talk about some trend. If we cannot reject H0, the data is not even conclusive enough to identify any "trend".

It's obvious that p=0.05 is an arbitrary cut-off that is not sensible in some occasions. But it is a convention - for a lower limit of the confidence in our interpretation (if we cannotreach this confidence, we dare not give an interpretation). This already is not a very high standard. We should be carful when interpreting results that are of marginal significance - but in the other direction, i.e. when p0.05 we very clearly should not anymore try to interpret the results.

Charalabos Antonatos , "Namely, simple interpetion of a non significant p-value in a clinical trial is not enough to exclude clinically significant effects" -- Absence of evidence is NOT evidence of abscence. The non-significant result onlymeans that the data are insufficient to even interpret the sign of a (possible) effect. The data are "not conclusive". There is no conclusion to be made, and it must not be interpreted as "there is no effect". This is a frequent mistake committed in many medical papers. I certainly agree that a look on the CI can helpt to figure out if the data are clearly incompatible with a clinically relevant effect. But, as I said before, this can also be tested directly by not using the "zero" H0 but using the "minimum relevant effect" as H0 in a significance test.

I think p-values could be flexible based on the sensitivity of the subject of study. If the research is about a general science, one can be more relaxed and go down to p=0.1. It is only in research activities that deals with human health and safety, that we need to be very careful about and be as conservative as possible on the p-values. As to my understanding, the final judgement is up to the researcher or research team.