Prior to going for equivalence test just confirm at .01 level with higher sample.

Rajesh Verma Yes, it's the same also at 0.1

The reality is that if you measured the temperature of all males and females, in whatever population you are thinking about, to a sufficient level of precision, there would be some difference in the temperatures. So even with equivalence testing, it's really not that there is no difference, it's just that there's no difference considering a certain threshold of difference that we are considering meaningful.

A couple of thoughts that help make the case that the difference in the two groups is actually small or unimportant:

• Present the mean values with confidence intervals about those means
• Present a standardized effect size (Cohen's d)
• Conduct equivalence testing
• Calculate the power of your test
• Make some argument about what a meaningful difference might be, perhaps in percent, or difference in mean temperature, or difference in mean temperature divided by standard deviation (which is Cohen's d).

Sal Mangiafico Thank you for your valuable support. I agree with you a slight difference must be present. I have to decide if I want to note it as meaningful or not.

I have a couple comments regarding your list, Sal Mangiafico.

1. In addition to the individual means and CIs, I would present the mean difference with its 95% CI. One reason is that the CIs for the two means can overlap even when the difference between the means is statistically significant. Second, I think that the mean difference is often more useful than a standardized measure of effect size, particularly when the DV is familiar to everyone and the meaning of a change of x units is well understood.

Article If we're so different, why do we keep overlapping? When 1 pl...

Article Standardized or simple effect size: What should be reported?

2. You included power analysis in your list. If you meant post hoc power analysis in the way it is typically done (i.e., computing power to detect the observed mean difference using the observed SDs), then I don't see the point. As Russ Lenth noted in the document linked below, retrospective power is a transformation of the p-value: When p < 0.05, retrospective power will be adequate; when p > 0.05, retrospective power will be inadequate. But perhaps you had something else in mind? Thanks for clarifying.

https://homepage.stat.uiowa.edu/~rlenth/Power/2badHabits.pdf

Cheers,

Bruce

You've used your p-value, it is gone. Can't use it again.

Thanks, Bruce Weaver .

On 1), I agree. The difference in means (and confidence interval) will be useful for the reader, especially as you note, when the reader is familiar with the units of the measured variable. ... I do like standardized effect size statistics. In this case, it makes sense to me to present the difference in means as a proportion of the standard deviation. Maybe that ends up being too wonky for most readers.

2) I'm working on wrapping my head around 2). I was thinking of retrospective power calculation.... I ran some simulations, and sure enough for a t-test *, a p-value of 0.05 corresponds roughly to a power of 0.5, and a p-value of 0.01 corresponds roughly to a power of roughly 0.8.

But the relationship isn't precise for all reps. ** If I have enough reps in the simulation, for a p-value of 0.05 the power ranges from, say, 0.5 to 0.90. And a p-value of 0.01 the power ranges from, say, 0.75 to 1.0.

So, I don't know. Practically, I don't know if calculating retrospective power has any use. Assuming I'm doing these simulations okay. If you do get a p-value near 0.05, and you calculate the power, if it comes out 0.9 does that really mean anything different than if it came out 0.5 ?

________________

* both groups with the same sample size, but a range of sample sizes, differences in the means and in standard deviations for either group.

** It appears that the spread in power for a given p-value is larger for smaller sample sizes, and actually the power is higher for small sample sizes. I'm not sure how to make sense of this. Maybe it's something to do with the power calculation function I'm using.

Report the (95%) confidence interval for the difference. This the the range of hypotheses about the (unknown true) difference that are statistically compatible with your observed data. If the largest values within this interval are parctically irrelevant, then your data is statistically sufficiently incompatible with any relevant difference, and you can say that (whatever the sign or the concrete size of the unknown true difference might be) the unknown true difference is not expected to be relevant. Otherwise your data is simply not sufficiently conclusive to say anything.

If you've already decided what the answer is doesn't statistics just get in the way?. I'm really disappointed to read some of the other answers that seem to tell him that it's ok to to use math to justify his preconceived notions with some mathematical contortions rather than an attempt to determine what Nature actually is. Best wishes David Booth

David Eugene Booth Actually by comparing the back temperature of 100 females and 100 males, there is no difference (according to the t-test) in the mean temperature.

During the past months, I asked here already how to statistically say that there is no difference between two observations and many people told me that I could not refer to the t-test's p-value > 0.05 as a term of "absence of difference".

David Eugene Booth aren't this two different question? Federico Roggio tested if the difference is incompatible with zero and did not find evidence for it. Now, he tries to figure out if the difference is compatible or not with any meaningful (I hope so) effect, which can be done by some form of equivalence test, TOST, ROPE or simply, as Jochen proposed, to check if the CI of the difference is sufficiently different from any meaningful difference. This may also help to evaluate if the information achieved with the sample is enough to reach a meaningful conclusion or not. I would not say per se that these are "mathematical contortions", but maybe I did not understand your point.

@Federico 200 backs don't necessarily give the answer for all backs. That's the point of probability. I can't talk about all backs unless I look at all backs with certainty. David Booth

@everybody I'm bowing out of this discussion because I have real problems to try to solve. But if all backs were at the same temperature all the time wouldn't that be nice because most infectious diseases would have been eradicated at that time. Best wishes to all and to all a good night. David Booth

If you have a good theory that suggests that the sexes *should* have different back temperatures, then consider the possibility of one or more confounding variables. Could temperature measures be affected by, for example, age or body fat? (Of course, these can also produce confounds.)

If you don’t have theory then stop. Move on.

Look up “Simpson’s Paradox”.

Anuraj Nayarisseri , do you really mean what you wrote or was it a typo?

EDIT: it seems that Anuraj deleted his post silently.

Hume F. Winzar Thank you, I liked your smart thought.

Jochen Wilhelm keep breathing man!

TOAST be tricky

Since you already have your result from the t test you might be tempted to apply an equivalence test but those should be applied only after you can define a region

I would just check the distribution and of the sample mean you have so you gauge what values are more plausible

Also Jochen, didn't you know?

P publications

P>0.05 means are equal -> dumpster

😅

Stefano Nembrini , since he has already used his 5%, and this finding is not independent of the test of another region, are you suggesting he still does this? Or are you saying that the person collects new data?