What are your opinion about using both the classical and modern psychometric test to develop a questionnaire?
Which one should be done first? And is there any implications in terms of the sample size?
Thank you.
Hi, Each method is suitable to its relevant context. For example, an already established instrument may need Confirmatory Factor Analysis in a new setting.
However, a newly developing instrument may go through the Exploratory Factor Analysis.
You might want to clarify what you mean by classical and modern psychometrics. Do you mean the IRT type procedures for classical or even older approaches like Cronbach's alpha. And for modern do you mean approaches for modern Next Generation assessments and serious games and other approaches to assessment. Also, are you referring to educational measurement, or other types of measurement?
Hi, Each method is suitable to its relevant context. For example, an already established instrument may need Confirmatory Factor Analysis in a new setting.
However, a newly developing instrument may go through the Exploratory Factor Analysis.
Hi, You could start, for the sake of simplicity, with Classical Test Theory (CTT). Go for Exploratory factor analysis (EFA) if you have no idea of how many constructs your scale measures, or Confirmatory Factor Analysis (if you have a theory for the construct dimensionality) - This will let you probe the construct validity. For construct reliability you can start wiht Cronabch alpha, but be carefull: while everybody uses alpha, it is a good estimate for reliability only if the items have equal factor loadings, otherwise you should use McDonalds Omega.
You can also use IRT, but this is more complex to manage and learn, specially if your construct is not unidimensional. IRT is quite usefull to evaluate the number of points required per item, as well as giving information on the discriminant abbility of each item and total scale.
Good luck (if you are starting in the less than consensual world of Psyhcometrics)
IRT brings more information to the table when building a questionnaire, I don't see arguable reasons to "regress" to CTT if you have the resources (sample size, item pool, software, stats skills, "open" reviewers, etc.).
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics