Hi everyone,
I am currently studying whether training against an AI bot on an online game can lead to bias mitigation. For that, I am performing a longitudinal analysis: participants will complete a questionnaire before training and a questionnaire after training (identical questions).
I want to explore changes in responses, but statistics are not my strong point... I already know that there is a debate over treating Likert scales as ordinal data or interval data.
Your help would be greatly appreciated! Thanks in advance.
Julia
There's no debate likert scales are ordinal. why not just rescale your likert scales to some convenient value and plot changes of response over time. This assumes that I understand your question. Best wishes David Booth
Hello Julia Accardo. Multilevel (or mixed) models are often used for analyzing longitudinal data. If you say what software you are using, readers may be able to direct you to some good resources. Meanwhile, here is a page with links you might find helpful.
http://www.bristol.ac.uk/cmm/links/
You may also find some useful info in this discussion started by @David L Morgan.
https://www.researchgate.net/post/How_can_one_help_people_create_simple_scales_from_Likert-scored_items/1
Good luck with your work.
David Eugene Booth I agree with the fact that Likert scales are ordinal. However, I have seen in the literature that some are treating Likert scales as interval scales... Thank you for taking the time to help!
There is a big difference between individual Likert-scored items and the use of multiple such items to form a scale. If you do have several related items that meet the criteria for creating a scale (e.g., they measure a single construct with exploratory factor analysis and they have an adequate level of reliability on coefficient alpha), then you can combine them into an interval-level scale. That scale can be used for correlations, regressions, etc. for over-time analysis.
@David L. Morgan I don't agree. OLS regression requires a continuous DV. This reminds me of the old Abraham Lincoln story punch line, " ... Calling a tail a leg does not make it so." Suggest you look at the definition of continuous variable. Any good calculus book should do. Best wishes David Booth
I have a clear notion of what a continuous variable is, and I certainly do not need a calculus text book to find a definition when just about any social science research methods text will do just fine.
I am hardly the only one who thinks a you can create a continuous scale from a series of Likert-scored items. And if it is good enough for peer reviewers, then that is the standard that matters most.
@David L Morgan one last thing. Stevens himself once said that measurement scales should not be used in statistics. They were only for psych. measurement use. Best wishes David Booth
I am not such what the purpose of psychological measurements is if you can't use them in statistical analyses. But as a query, would you accept using confirmatory factor analysis and structural equation modeling for the measures in this question? And if so, then what are the differences in the assumptions between those methods and constructing simple scales.
This simulation study may be interesting to followers of this discussion.
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological methods, 17(3), 354.
https://web.archive.org/web/20160309162825id_/http://psych.colorado.edu:80/~willcutt/pdfs/Rhemtulla_2012.pdf
I'll add one other comment. IMO, most of the models we use in data analysis are approximate. Even OLS models with continuous outcomes that have true interval (or ratio) scale properties are approximate (assuming that the error distribution is not perfectly normal and perfect homoscedasticity is not achieved). From this point of view, it seems to me that an outcome variable with approximate interval scale properties might very well suffice. But of course, there can be disagreements about what conditions have to be met to make the approximation good enough to be useful. Simulation studies can be helpful in answering that question, I reckon.
Article Resolving the 50-year Debate Around using and Misusing Likert Scales
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation