Ignoring the stress results you see when testing more than once per day, the ABBA study will require somewhere between half as many and just as many human subjects as if you ran the AB study on different days. You, or a professional biostatistician working for you, could produce estimates for the required number of human subjects for your next trial based on the results of your pilot trial. Given that you have observed negative effects on your test results when running two experimental interventions in one day, I would recommend that you not run two interventions in one day during future trials.

By the way, I hope you just oversimplified your experimental design in your introduction. Systematically running your interventions in the order of A1, B1, B2, and A2 will confound the effects of the intervention with the order in which the interventions are given. If you run your future trials so that it is equally likely for an individual subject to receive A1 followed by B2 as it is for that subject to receive B1 followed by A2 then it will be easier to tell if the effects you see are caused by the specific treatment or by something that happened differently on the different days.

I wish you success with your research.

Chuck

I'll check how many test person AB would cost, when replacing ABBA in this study...

Is there any test to check if ABBA give at least other results than AB? I mean, with my collected data both design could be evaluated. Wilcoxon tests will fail for sure, I guess...

This design was chosen on recommendation of the manufacturer. But this is a general recommendation, so not specific for this study.

The problem I see is that every test person at the first day gets tested with the settings the person is used to (A1). Then I modify the devices software and test this alternative setting right after switching to it (B1). After 6 weeks of getting used to the alternative setting, at the second test day, the alternative setting is tested first (B2) and the old setting after that (A2).

When looking at every second test of each day, at B1 (compared to B2) every person apart from excellent "performers" will reach only a low score (bad performance). But the corresponding test A2 relatively to A1 won't be that low because the test person was used to this setting for min. 6 months before the first test day. So this will result in a score not a good as the A1 score but much better than B1 (both relatively to the other test with the same setting (A or B)).

PS: I hope this isn't too confusing for an person not involved in this study...

thank you and best regards,

Daniel

Hello Daniel,

The reason for using a crossover design is so that each subject is their own control. You could do the same experiment with different subjects receiving only a single treatment but it could take a lot more human subjects than a crossover experiment. However, the crossover really only works if there is no 'crossover effect.' Your stress problem is a 'crossover effect.' It sounds like you know how to eliminate the effect but there are costs involved. Put six weeks of training between your tests and the immediately obvious crossover effect goes away. Now, all you have to do is fund the training and find volunteers willing to put in an extra twelve weeks.... Of course, you will have the data to see the strength of your crossover effect and you can think about whether you want to do AB or ABBA.

With regard to a Wilcoxon test, his signed rank (as opposed to rank sum) test would work fine with an AB design where the order of treatment is determined at random. Under real world conditions, a Wilcoxon test will be more powerful (more able to determine a statistically significant difference with a smaller sample size) than a t-test.

Again, I wish you success with your research.

Chuck

Hello Chuck,

I'll have 3x ABBA and 8x BAAB at the end of my current work. I know how to compare A and B but what I don't know is how to evaluate ABBA|BAAB. How do you evaluate data of ABBA|BAAB? Is it even possible with 3x ABBA and 8x BAAB data sets? Or is there no special methode when you have ABBA|BAAB instead of another COD? Do you always "just" compare the arithmetic means of A and B? So this data just already is ABBA|BAAB and Wilcoxon just gets higher class data because of a higher class COD?

thank you very much,

Daniel

It is possible, and routine, to test for carryover effects. Note that having data with more of one treatment order than the other ("unbalanced" data) will probably reduce the statistical power of your tests (reduced ability to obtain 'statistically significant' results in comparison to the same number of samples where the data are "balanced"). I'll summarily describe how to do this with an AB/BA study and then with an ABBA/BAAB study.

With an AB/BA study, you test for carryover using a two-way analysis of variance (ANOVA) or a nonparametric equivalent (e.g., Friedman Test). Your first factor is treatment and your second factor is order. If the factor for order meets your criteria for statistical significance then you decide that you have a carryover effect.

With an ABBA/BAAB study, there are options. Off the top of my head, I'm not sure if some of those options are limited by the unbalanced data. The easiest option would be to run another two-way ANOVA or equivalent where you redefine what you mean by order and then use some family-wise adjustment to obtain test-wise p-values for your A/B, AB/BA, and ABBA/BAAB tests.