Hi all,
I’ll paste the more specific version of the question below before unpacking what I’m asking:
In psychometrics, what are some established quantitative methods to examine competing operationalizations of the same construct that all purport to predict the same outcome variable, to test which one is most efficacious at predicting said outcome variable?
Though I have experience with quantitative analysis/statistics, I’m relatively new to psychometrics (as it isn’t my area of expertise). Though, I understand construct, convergent, discriminant, face validity, and so on, of course.
I’m specifically asking the above question because have several different operationalizations of student motivation from various sub-disciplines across the social sciences. Many of which purport to predict “real-world” measures of student achievement (e.g., standardized midterm and final exam). Is there a best practice to determine which one of these is the most valid predictor of these outcome measures?
I know to test discriminant validity, researchers often have participants take similar scales and then use EFA to examine the factor loadings (I realize I’m simplifying here). However, given the number of scales and items, ideally, what I’m trying to avoid is having the entire sample in such a study (i.e., students) take these measures all at once, especially since the scales are relatively similar.
Given that this is exploratory in nature, could a potential way to compare these scales be to randomly assign students to take the various measures of student motivation mid-semester. Then, regress the different scales as predictors of final exam grades to see which scale accounts for the most variance?
I understand that researching student motivation and student achievement are much more complex; this is just a hypothetical study idea given my minimal knowledge on the topic.
I would be grateful for any and all insights.
Thank you!
Probably a good place to look is item response theory (IRT), though whether this is a "theory" of the response process is questionable. Like EFA, IRT is a latent variable approach and both are path of the general linear latent variable model. Depending on how happy you are with stats see Book Latent Variable Models and Factor Analysis: A Unified Approach
. When I was a psych grad student (long ago) Bartholomew was one of my professors so this was discussed. There are lots of newer approaches and older approaches/models too. I'll let others give examples.
Most literature I have read, they always begin with EFA, then CFA or CCA. I have also seen some recommendations to do parallel analysis to strengthen the dimensions established in EFA.
hi,
please have a look at the following links:
Article The Role of Psychometrics in Individual Differences Research...
Chapter Introduction to quantitative methods
https://dissertation.laerd.com/constructs-in-quantitative-research.php
hope it is helpful .
best wishes
- Badges
- Science topic