Consider a within-subjects study where each participant executes tasks (5 levels) in each visualisation (3 levels). As such, there are two independent variables: the task and the visualisation. For each task, the experimenter annotates the accuracy of the participant (the dependent variable) which is a binary value: correct or incorrect. This means that all variables are nominal. Which statistical test should the researcher use to check whether there are interaction effects between the independent variables on accuracy as well as main effects?
Since the dependent variable is nominal, I do not think a two-way repeated measures ANOVA can be used. Converting the accuracy to 1 (correct) and 0 (incorrect) would allow the researcher to use the ANOVA, but I don't think the conversion would be an appropriate approach to study the relationship. A Chi-square test of independence (R x C) cannot be used as well, since it supports only two variables. The only test that test that appears to be reasonable to use it the multinomial logistic regression. Any opinion on the test that should be applied? Thanks in advance.
No, you can't use traditional anova. Because your dependent variable is binary, you have the relatively easy option of using logistic regression. A good software implementation of this will allow you to explore the interaction and do post-hoc analysis.
There are two questions needed to be addressed first before you decide which model to be used. (1) How many participants were recruited in the study (2) Do you expect there will be systematic bias related to participants (some participants are more likely to get all 15 questions correct than others).
When the outcome is binary, logistic regression (GLM with logit link and binomial family) is the appropriate method and the interaction effect can be easily evaluated. However, you also have repeated tests nested within the same subject. So the correct method is GLMM (https://mran.microsoft.com/snapshot/2018-02-25/web/packages/glmm/vignettes/intro.pdf ), in which you model that some participants are more likely to correctly rate all tests. When you have many participants, you can model this "systematic" bias using random intercepts and you can also include other factors that might impact the participant's ability. If you only have a few participants and it is believed that there is minimal "systematic bias ", then you can potentially use the simple logistic regress.
I absolutely agree with @Caroline: since "you also have repeated tests nested within the same subject,...the correct method is GLMM. " .
Lastly, you wrote "Chi-square test of independence (R x C) cannot be used as well, since it supports only two variables," but there is a variant of Row Column Layers (RCL, additional stratification), including the nested levels, in analysing categorical data. A three-way contingency table is a cross-classification of observations by the levels of three categorical variables, and generalizing, a k-way contingency table is used to classify observations(counts or percentages) by levels of k categorical variables.
However, If your categorical variables are of "pre-test" and "post-test" observations case, definitely the chi-square test of independence is not appropriate as the assumption of independence of observations is violated, rather a McNemar's Test should be the most appropriate.
Logistic regression is supposed to be the usual starting point when your dependent variable is categorical.
You know, ANOVA is nothing but the same model as linear regression and the dependent variable is continuous.There are others types of regression models depending on the characteristics of the dependent variables; binomial logistic regression is used when your DV is dichotomous, multinomial logistic regression used for DV which has more than two unordered categories. Another version of multinomial logistic is ordinal logistic, but, in this case, the categories of DV are ordered.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA