Dear all,
As a part of my phd work, I plan to construct an assessment tool for a clinical population using a mixed method sequential exploratory design (I plan to use a grounded theory approach to build a theory, based on which the instrument will be developed). The instrument that I plan to develop is going to be a 45-60 minutes semi-structured interview which will have standardized scoring.
Since the tool is targeting clinical population and is quite lengthy, it won't be possible for me administer it on a large sample in the quantitative phase (planning to recruit around 100). And this might further cause a challenge in running Factor analysis for validation.
1) I wanted to know what could be some other ways to establish validity with such limitation.
I will be establishing content validity in the first phase itself. I am not administering any other standardized tool due to paucity of time and resources.
2) can factor analysis be done for a small sample i.e. within 100?
3) If not, then can I just establish reliability, and content validity and skip construct concurrent validity all together?
Thanks!
Hello Shikha,
You can always use an anchor test design (e.g., everyone is administered a common set of items, then some subset of the remaining items may be unique to one respondent or subset of respondents) and item response theory to obtain information about the dimensionality, consistency, and behavior of the items within your theoretical structure. Doing so means that you don't overly burden any one respondent with excessive task demands, and still you can gather sufficient information to get started on your quest. However, the number of respondents you'd like to have per item is generally 100-200 as a lower bound; more if you wish to try a two-parameter IRT model or higher.
Factor analysis with 100 cases would be plausible if you were working with a very small number of items/questions/stimuli/variables. However, if the set is as large as your query suggests, you'd have to consider a larger N to get stable estimates about whether a given structure performs well to reproduce the observed relationships among the pairs of item scores.
If you are simply unable to collect additional information, and are willing to believe that your data are essentially unidimensional, you could apply classical item and test analysis statistics (e.g., means, item-total correlations; estimated internal consistency reliability) as a starting point. As well, remember that computing and presenting correlations of your measure with scores on related (or unrelated) variables can help you argue for convergent (and discriminant) validity, without having to engage in factor analysis.
Good luck with your work.
David Morse that's a unique perspective! thank you for your suggestions. I am going to read up on that
Technically speaking, factor analysis is a way to establish reliability rather than validity. For construct validity, you need to test a predicted pattern of correlations, including other items where you expect positive correlations (convergent validity) and negative correlations (divergent validity).
With regard to factor analysis as a way to determine how many dimensions your items measure, I agree that a sample of 100 is not likely to be enough if you have a large number of items.
Dear Shikha Ahuja by asking this question once again with some alteration it sounds like you already have made your mind and want confirmation from RG community. Anyways, although you mentioned about developing a theory through grounding theory method and, based on it, want to develop an instrument with some mixed method approach, you directly jump to reliability/validity issues. However, I think most important is that how you develop a theory, definitions, and pool of items. It might be a daunting task for a PhD scholar. Try to read the following article to have some idea on how to develop a theory and content in the first phase. Tq.
Kahn, W. A. (1990). Psychological conditions of personal engagement and disengagement at work. Academy of Management Journal, 33(4), 692-724.
- Badges
- Science topic
- Similar topics
- Instrumentation Engineering
- Instruments