Hi everyone.
I am having a mental blank, as I’m struggling with sleep deprivation from a teething AND sick toddler at the moment and just want to double-check my approach to my data analysis is correct before I begin. Also, if anyone has any articles/references relating to the statistical validation of a measurement scale that has been refined using Rasch modelling, that would be wonderful, as I’m always keen to find more resources to ensure I thoroughly understand my subject-matter.
So, here’s some context:
I’m developing a new measurement scale for my PhD. I’ve so far used Group Concept Mapping to examine the topic and build the questionnaire; I’ve followed this up with cognitive interviewing to verify the face validity of each questionnaire item; I’ve then used Rasch modelling/Rasch analysis to refine the questionnaire and verify that the response scales are functioning as expected, that each subscale is unidimensional, etc. etc. and now I have finished collecting my data for the validation of the scale. My scale has nine subscales, each with a minimum of seven (7) items. Comparison scales exist for all but one of the subscales. I’ll also be looking at sex differences (M/F) and age cohort differences (midlife/later-life).
This is my current analysis plan, and I’ll be using SPSS for these analyses:
- For all;
- For male vs female; and
- For midlife vs later-life
Have I forgotten anything? Or are there better approaches you’d recommend (and why?)
Article Associations Between Sleep Quality and Nursing Productivity ...
I do not know Rasch modelling/Rasch analysis well.
Thank you for your answer Dr Mergen. I think that you may have misunderstood my question, however.
The scale and subscales have been designed using Group Concept Mapping (GCM). In this instance, GCM takes the place of the EFA, as it uses multi-dimensional scaling and heirarchical cluster analysis to identify the different item clusters (equivalent to factors in this case). The items were developed from the statements collected as part of GCM, and then verified for clarity, relevance, etc. through cognitive interviews with a sample of 30 participants. These items were then used in the pilot study.
The pilot study has already been conducted with 181 participants, and I used Rasch modelling to refine the scale and subscales. This includes verifying unidimensionality of subscales, point-measure correlations to assess rating scale function, person separation index (In Rasch analysis, the PSI is comparable to Chronbach Alpha, KR-20) to determine reliability of the subscales, examination of item fit against the Rasch model, person-item distribution scores to examine item/subscale discrimination abilities between high/low scorers, and DIF analyses to check item functioning across different demographic groups. All that is left now is to verify the traditional reliability and validity of values of the finalised scale. May I assume from your answer that I have proposed is reasonable?
Dear Dr. Haluk Mergen
,I agree with you because there is a difference between the existing scale and its psychological properties are checked, and the scale is set from zero.
Dear Dr. Ashley Macleod ,
Content validity is an assessment of how well a set of scale items matches with the relevant content domain of the construct that it is trying to measure.
A measure can be reliable but not valid, if it is measuring something very consistently but is consistently measuring the wrong construct. Likewise, a measure can be valid but not reliable if it is measuring the right construct, but not doing so in a consistent manner.
Haluk Mergen
and Abdelkader Mohamed Elsayed . I appreciate your responses.Can I confirm that the following approach would be appropriate?
1. single factor CFA of each subscale to identify how each item loads onto the factor initially to help determine whether any misfit occurs at the subscale or full model level
2. two-factor CFA of each pairwise combination of sub-scales to determine if and/or where any subscales overlap (i.e. cross-loadings and cross-factor correlated residuals)
3. full factorial model of all subscales to determine local fit and global fit of items and subscales to model.
If I have HIGH correlations between factors in either the pairwise or the full model, this would indicate a lack of discriminant validity between subscakes, and if I have LOW correlations between items and subscales, this would indicate a lack of convergent validity, is this correct?
For this scale, all of the subscales are designed to measure components of a single, overarching theme (sexuality/intimacy), so I expect a degree of cross-loading and/or moderate correlations between subscales.
Have I missed anything, or have I misinterpreted your feedback?
Once again, thank you for taking the time to respond.
Regards,
Ashley Macleod