I have conducted an experiment where 31 testers were driving in an autonomous driving simulator to research the feeling of security and transparency during autonomous driving decisions.
Testers were asked 6 Likert scaled questions (5 steps, strongly agree to strongly disagree), which were divided in 2 groups (3 questions per group): feeling of security and transparency.
These 6 questions were asked 6 times, for different types of feedback: no feedback, light, sound, visualization, text and vibration.
All testers were testing all 6 types of feedback. As I see it, that means that I have within subjects study design with 6 groups of answers (no feedback, light, etc.)
I now want to compare the no feedback questions with the 5 other types (light, audio, etc.) to see if the different feedback types increase the feeling of security and transparency compared to no feedback.
E.g. no feedback vs. light, or no feedback vs. audio.
I am thinking of using a T-Test or Mann-Whithney U-Test to make this comparison. Furthermore, I would calculate a Cronbach alpha value for the 6 groups (no feedback, light, ...).
Is that a approach for a small sample size of 31 testers? I am not sure if T-Test and M-W-U are the correct way to go to analyse my Likert data.
Thanks,
Toby
Why not ANOVA with appropriate post-hoc test ? Or Kruskall Wallis ?
There are some interesting discussions in other places already on this. I am picking two interesting views to help you get 'confused' more for your reading benefit :)
https://www.researchgate.net/post/Is_a_Likert-type_scale_ordinal_or_interval_data
https://statisticsbyjim.com/hypothesis-testing/nonparametric-parametric-tests/
Regards,
Akilesh. R
Multigroup factor analysis to evaluate differences in invariance, can also perform MIMIC analysis that allows using the prediction of categorical variables (groups, age, sex, etc.) to determine the heterogeneity of the model considering these variables.
Akilesh Ramasamy thanks for your response! I have non parametric data, so for more than 2 groups it would be Kruskal Wallis, I agree. But in the end I'm always comparing only two groups, aren't I? For example no-feedback vs. light, or no-feedback vs audio. That's why I was thinking about the Mann-Whitney U-Test.
Cristian Ramos-Vera thanks, I will have a look at MIMIC.
Tobias Schneider Yes. During post-hoc analysis we will have to apply adjustment to control for inflation of Type I Error, using either a Bonferroni or Dunn-Sidak correction to the significances obtained from the series of Mann-Whitney test results.
To Note:
Cristian Ramos-Vera mentions about two techniques
1. multigroup confirmatory factor analysis (MGCFA)
2. MIMIC
Both may be used in your situation, is the suggestion, if I understood right!
More info at:
https://www.ibm.com/support/pages/post-hoc-comparisons-kruskal-wallis-test
(Regarding post hoc adjustment)
http://www.statmodel.com/discussion/messages/9/114.html?1564446240
(Regarding MIMIC & Multigroup CFA)
~
Akil
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis