Good afternoon,
I do have a question about my thesis project. I am estimating predictive validity of an assessment, however, I am a bit confused. Assessment consists of major factors,which in their turn consist if smaller dimensions which in their turn consist of smaller constructs. I need to formulate hypothesis but I do not know which levels of the assessment they should include: major factors, dimensions or smaller constructs?
Thank you!
Hi Maru,
I study EFL reading construct. To estimate its predictive validity, I developed first an Expert Reading Construct based on monolingual literature. Then I developed a reading test and administered it in a sample of 111 Chinese EFLs. Results were analysed using Rasch Modelling. Based on their logit measure in the IRT analysis, I re-arranged item difficulty level and compared it with the Expert Reading Construct. An EFL reading construct is thus built, and it has a predictive power regarding Chinese EFLs.
Hope this helps.
Regards
Orieta
Hi Maru,
This is a hard question and one that I don't think you will be able to get a concrete answer to on ResearchGate, at least without being very specific about the construct, dimensions, and outcomes you'd like to predict.
I suggest you schedule a meeting with your mentor and discuss the problem. Be prepared for an answer that consists of suggested readings rather than specific hypotheses.
Here are some other answers that might work for you:
(a) If this is an established instrument, read the original publication carefully for statements by the author(s) to determine the intended use of the instrument, which should implicitly or explicitly include statements about how the numbers derived from the tool should be useful for predicting outcomes.
(b) In my experience, it is better to formulate your hypotheses involving broader domains than specific domains. In the first case, if all of your factors, dimensions, and constructs are positively correlated, you will be unlikely to be able to detect specific effects attributable to the different aspects of the measurement. And even if you do detect differences, they are unlikely to be reliable or reproducible.
Good Luck
Rich
Hi there,
Your decision should be driven by theoretical reasons. What is the most important theoretical question you want to answer?
It is good to note that criterion-related validity is a function of conceptual overlap between the predictor and criterion, as well as predictor- and criterion specificity (i.e., how broadly you wish to measure the constructs). For a discussion of this topic, please see a personality paper by Soto and John (2016) with the title "The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power" in JPSP. Good luck!
R. Jones made the key point: "if all of your factors, dimensions, and constructs are positively correlated, you will be unlikely to be able to detect specific effects attributable to the different aspects of the measurement." That means you will probably need to do your predictive validity tests at the highest level -- factors, rather than their constituent dimensions and constructs. It seems unlikely that the theoretical framework you are using would predict different outcomes for the different dimensions or constructs of any given factor; the theoretical predictions are probably made at the level of the broad factors. If you do have theoretical predictions at the level of dimensions or constructs, and if the dimensions of a single factor (or the constructs of a single dimension) are not themselves highly intercorrelated, then certainly you should test those predictions But as a practical matter, most of your tests will likely be at the factor level.
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics