I always thought that validity testing was necessary. However, when reading journal articles, I noticed that many of them do not report the validity of their scales. Is it because they omitted this step due to space limitations, or was it simply unnecessary in the first place?
If validity testing was conducted, and the original scale contained more than ten items, did all factor loadings meet the standard (which is quite difficult in practice)? Yet, in these papers, not a single item was removed.
I would greatly appreciate your valuable insights on this matter.
I would say it depends on (1) the extent to which the scale has previously been validated and (2) whether previous validation studies apply to the population examined in your study. That is, if the scale has already been shown to be valid in dozens of studies and you examine a similar population of individuals, it may not be necessary to "re-validate" the scale. On the other hand, if you are studying a different population, it may be necessary to examine the scale again. In any case, it can never hurt to factor analyze the items to check their psychometric properties, provided your sample size is large enough for a meaningful factor analysis.
Thank you very much for sharing your valuable expertise! If I may seek your advice once again, I would sincerely appreciate your insights on the following question: If a study utilizes all the items in a scale, the overall reliability is high. However, when conducting confirmatory factor analysis (CFA), some items are found to have low factor loadings. In this case, should the items with low factor loadings be removed? Since removing too many items may lead to a decrease in reliability, would it be acceptable to retain all items despite their low factor loadings, given that the scale has already been validated? What is the more standard approach to handling this situation?
My position is that CFA should precede any scale/composite reliability analysis. That is, you first need an appropriate factor model before it makes sense to figure out which reliability coefficient is even meaningful and interpretable. "Appropriate" here means the factor model should fit well (not easy for a single factor model with many items), and the items should have decent standardized loadings/small standardized error variance components. If items don't load substantially, this likely indicates scale validity problems (i.e., the items don't measure the factor in question and/or measure other specific factors). It does not make sense to compute single-factor composite reliability (e.g., Cronbach's alpha) across a set of items that are partially invalid or multidimensional. In summary, a proper CFA measurement model should be established first. Then you can determine the scale/composite reliability coefficient that aligns appropriately with the CFA measurement model.
Depending on the established items if they are valid and reliable from reputable journals. My position is CFA. My opinion and all the best
If you're using an established scale, you generally don’t need to conduct EFA. However, performing CFA to confirm the scale's factor structure in your sample is recommended.
- Badges
- Science topic