In attachment a R script wich might be helpful. First it is tested whether the 2 groups differ in types of reponses over the 3 times. Then the Cochran Q test is run for each group. Then 2 non-parametric tests are included.

I would argue that a more direct test is the regression form of your ANOVA E.G see: Applied Linear Statistical Models | Michael H. Kutner, William Li, Christopher J. Nachtsheim, John Neter | download (b-ok.cc)

Best, D. Booth

Dear Jos Feys , thank You for that script. This is a very interesting approach, but I have some doubts. The Fisher test and its use in the script indicate that factors with repeated measurement are treated as between-subjects. Further tests used also indicate that the database is transposed and treated as if it were an intergroup measurement.

Of course - the proportion for data and further interaction components can be calculated in this way, but I have doubts as to whether it is methodologically justified to transpose repeated measures data (within-object factors) to groups (between-subjects), while the methodology usually indicates the need to use other tests (hence, for example, Cochran's Q usage). Is there any justification in the literature that I could use when publishing the results obtained in this way?

Dear David Eugene Booth - thank You for answer and book mentioned, but what exactly do you mean about regression? In terms of - how to specifically relate to dichotomous (nominal) repeated measures through regression?

In attachment a better script. The Fisher test does not imply I treated the repeated measures as between subjects factor. I tested for the differences in patterns of answers between the different Groups. I fabricated data to reveal an interaction. The separate Cochran tests indicate Group 1 yield sign. p values, wheras Group 2 does not. The non-parametric tests indicated the interaction between Groups and Time (repeated measures) is significant. I think you can report such tests.

@Tomasz RAK

A logistic regression is also called a generalized linear model A repeated measure is a commonly used experimetal design generally called factorial(which has a very easy linear model RHS) see. Kutner et al or: https://www.google.com/search?client=firefox-b-1-d&ei=GU_IX4mcOqrU5gKuoZagBg&q=repeated+measure+as+an+experimental+design&oq=repeated+measure+as+an+experimental+design&gs_lcp=CgZwc3ktYWIQAzIICAAQ6gIQjwEyCAgAEOoCEI8BMggIABDqAhCPATIICAAQ6gIQjwEyCAgAEOoCEI8BMggIABDqAhCPATIICAAQ6gIQjwEyCAgAEOoCEI8BMggIABDqAhCPAToHCAAQRxCwA1CtjE5Y6MZPYJLMT2gCcAJ4AIABAIgBAJIBAJgBQKABAaoBB2d3cy13aXqwAQnIAQjAAQE&sclient=psy-ab&ved=0ahUKEwjJjPfH47DtAhUqqlkKHa6QBWQQ4dUDCAw&uact=5

or https://www.google.com/search?client=firefox-b-1-d&ei=WlTIX4GzKILgxgHN_IbYCg&q=repeated+measure+as+a+regression&oq=repeated+measure+as+a+regression&gs_lcp=CgZwc3ktYWIQAzIJCAAQyQMQFhAeMgYIABAWEB4yBggAEBYQHjIGCAAQFhAeMgYIABAWEB4yBggAEBYQHjIGCAAQFhAeMgYIABAWEB4yCAgAEBYQChAeMgYIABAWEB46BwgAEEcQsAM6BQgAEM0COgQIIRAKOgYIABAIEB5KBQgHEgExSgUICRIBMkoHCAoSAzE2NVDD8AxYzscNYLPPDWgBcAJ4AIABiQWIAc0nkgEMMC4yOS4xLjEuMC4xmAEAoAEBqgEHZ3dzLXdpesgBCMABAQ&sclient=psy-ab&ved=0ahUKEwiByJHJ6LDtAhUCsDEKHU2-AasQ4dUDCAw&uact=5

Which is quite easy to run in anything.

Best wishes,

David Booth

PS sorry for the long links

You might also look at:

Design and Analysis of Experiments, 9th Edition | Douglas C. Montgomery (Author) | download (b-ok.cc)

PPS in case I have not been clear your ANOVA is also called a logistic regression with binary dependent variable. As best as I can tell this is what you mean if your DV is dichotomous (Success, not success) which you never actually say. BTW interaction terms are of the form x1*x2 and so interaction tests are just another t-test on your regression output. At some point statistics professors will learn to say regression and ANOVA are two ways to do the same calculation, demonstrated by Fisher here: https://royalsocietypublishing.org/doi/10.1098/rstb.1925.0003

I added the logistic regression as suggested by David Eugene Booth.

In attachment the updated R script.

I also added a Bar Plot based upon the R Companion:

https://rcompanion.org/handbook/H_07.html

and the Post Hoc Pairwise comparisons after the Cochran Q test for Group 1

I added a facet bar plot.

Dear Jos Feys and David Eugene Booth , thank You for Your answers. I still got some doubts for that ideas.

First of all - i understand that i can change this experiment design, but ideas that You gave treat data as between-group and independent data. In the literature - for regression like anova or chi based plans there is a clear separation between plans that have independent data (separate groups) and plans that have repeated measures. In other words - if I have repeated measurements (and yet I do), the data is not independent, so changing the plan through the analyzes you propose will give different results than the analysis with repeated measurement. Is there any justification in the literature that I can treat interactions with repeated measures as independent interactions (without violating the assumptions about testing the null hypothesis)? Fisher's article from 1925 David cited talks about independent data.

Second - what about arguing, that repeated measures regression isn't generally a very good model (e.g., because it reduces degrees of freedom estimating slopes for each "person" to "database obdervation"). A similar argument often applies to the "division" of the database into "subgroups" - there is an argument that this is not a determinant of the interaction effect, eg for ANOVA, but two separate analyzes that test the hypothesis only on some observations from data. The chances of making both kinds of errors just increase.

Third - main/general question here is still did not receive a reply - if there is an equivalent interactive test for Cohran's Q - up to this point has not been given here.

And last thing - I understand that I can, in principle, use any analyzes at this level, freely change the research plan or transform the data... but I would prefer at the moment of publication or even the reviewer's doubts or my own beliefs being able to quote a specific author who proves that the method is acceptable.

Thank You, and regards

In the script, I first tackle the question whether the 2 groups differ in pattern of 0/1 choices made. The chi-square test for the differences in patterns between the 2 groups, in this example is significant: p-value = 0.02195

-----------------Pattern -------- Group ----0-1-2-3-4-5-6-7

----1------- 2-7-2-1-3-1-0-0 ----2------- 0-1-2-2-6-2-4-3

The data were so fabricated to yield this difference in pattern. This significant difference in numbers of patterns shows there is an interaction between groups and repeated measures (0/1 choices). Given this interaction, the second question is whether the Cochran Q tests for the separate groups yield different results. It turns out the Q test is significant for group 1: Q = 9, df = 2, p-value = 0.01, but not for group 2. The non-parametric tests corroborate the significant interaction.

If you would like to double the repeated measures, e.g T1, T2, T3, T4, T5, T6, you could run the Cochran Q test followed by post hoc McNemar tests (thus, adjusting for multiple p-values). Instead of a (2x3) Between x Within subjects design, you then could test a (3x3) Within x Within design.

Repeated measures are now called longitudinal models. Read class notes from one of the inventors. Very important for biomedical stats.

Davidian Notes-current:

https://www4.stat.ncsu.edu/~davidian/st732/notes.html

Best wishes, David Booth

OK, after a long review of the available material I will summarize what I was able to establish in the literature together with citations.

First of all (you can skip this paragraph, because they are methods that give "a result", but without any justification in the literature), what I did as basic/simplified methods - I the Cohran Q test for the total number of cases and in subgroups, with the Dunn-Bonferroni post-hoc test in all cases interpreting post-hoc differences between subgroups as an interaction effect; however, I cannot prove that this is the correct approach. Then, as suggested here, I performed proportion tests based on logistic regression (Chi-square test with logistic regression correction, available from SPSS/Statistica/R); inter alia, separately for positive results proportion and negative results proportion. I did not get any significant results here.

I decided to simply perform a mixed ANOVA, i.e. establish the research group (season) as an between-subject factor, and the results of the method as an within-subject factor (aka repeated measurement). So the design experiment I have, as mentioned, is: 2 (season) x 3 (drugs) (repeated measurement) and the dependent variable is / are a dichotomous nominal variable.

The dichotomous dependent variable (3 dependent variables in the repeated measure plan in this case) can be interpreted as follows: the mean calculated from the indications is the same as the percentage of positive indications. For example - 4 negative indications (0) and 6 positive indications (1) give the average of 0.6 or 60% positive indications and this is how the mean can be interpreted.

Why this can be best method to obtain information about interaction here? Although the idea of ANOVA with a dichotomous dependent variable has been known for a long time (because ANOVA to some extent will meet its criteria with such a variable, see Glass, 1972), different researchers (see for example Jaeger, 2008) indicate that the results of analyzes based on this approach may be biased. Long story short: ANOVA results are based on standard error estimation, and this for binary variables may not be calculated correctly. BUT! Lupsen (2018) conducted a very extensive simulation research on alternative methods of statistical interaction research with binary dependent variable, checking the correctness of the obtained results using various methods. This publication (although quite complex) is very valuable here, because it shows: yes, ANOVA may give erroneous results, but the other alternative methods in the mixed design scheme do not give more correct results. Simply put: if an alternative test "protects" from some sort of error, it is "worse elsewhere". Especially in the case of, for example, unequal compared subgroups. Note (!) that the author recommends using the Huynh-Feldt correction in this case! Of course, the regression mixed models mentioned in the previous comments and many other methods were also tested here. The undoubted advantage of this method is the simplicity of implementation, for example in SPSS, Statistica, or R, and most importantly - a very simple calculation of post-hoc tests for interactions, which in the case of other "equally good/bad" methods can be very complicated, or even unavailable for average scientist.

That's all. And here's specific literature in which the theses are developed:

• Lüpsen, H., & Rechenzentrum, R. Anova with binary variables-Alternatives for a dangerous F-test. Universität zu Köln.
• Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of memory and language, 59(4), 434-446.
• Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972). Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of educational research, 42(3), 237-288.
• and here's some interesting discussion about simulating data and comparing other methods for binary data: https://stats.stackexchange.com/questions/340129/anova-with-binary-dependent-variable

Good luck to anyone reading this :)

I added the Mixed ANOVA (2 Between x 3 Within) using the ezAnova from the ez R package, which includes the Sphericity corrections. This ezAnova also includes the ges = generalized Eta squared η2 effect size. I chose the type 2 Sum of Squares because the design is unbalanced.

I also included the paper by Haiko Lüpsen on R-functions for the analysis of variance, and a paper on the nparLD R functions.