Rainer Duesing ,

I read through the thread and the Kaiser paper.

I don't know if it's just me, but this all seems so silly, with one exception, I'll mention at the end.

It seems very silly to me to say, "When you get a significant two-sided t-test, you know the means are different, but then you can't look at the actual means to see which is bigger."

Is this how anyone uses hypothesis testing in the real world ? If so, we could never make a practical decision about anything... "We say there is a significant slope, but we can't look to see if the slope is negative or positive, or what the magnitude of the slope is." That logic leads to great insights.

Other than this silly-in-my-eyes "logical" point... For both the Significance paper and the Kaiser paper, it seems to me that the advantage the authors point to is that you could use alpha = 0.05 for each of the directional tests, which gives you higher power than using alpha = 0.05 for a two-sided test, but each hypothesis retains the alpha = 0.05 error rate.

This just seems to ignore that the fact the error rate for both directional tests together would be greater than the nominal alpha.

Both papers address this somewhat, but in neither case do I feel like they address this point clearly or practically.

I could be wrong.

The one thing I found interesting in the Kaiser paper is that they raise the possibility of using two different alpha values for the two different directional tests. This could have practical implications. Like you want to be really sure a new treatment is better than the standard treatment, but would buy weaker evidence that a new treatment is worse than the standard treatment.