I'm at the end of my wit and much appreciate the help from the community. I'm conducting a confirmatory factor analysis (CFA) to test whether two factors — frequency of experience and intensity of the experience — adequately measure the construct (discrimination).
Both factors have the same set of items. That is, each item is rated twice: once for how frequently the experience occurred and once for how stressful it was perceived. It may sound weird, but I'm adapting a test here and just following the construct of the original one.
Given this setup, the two factors are highly correlated (which is expected since they use the same content but assess different dimensions). This high correlation may violate CFA assumptions (e.g., local independence). Therefore, I’m considering using a bifactor CFA to model a general perceived discrimination factor and two group factors: one for frequency and one for intensity.
My questions are:
I have talked with my advisor about this, and she has confirmed that the traditional CFA is unsuitable here since the correlation between frequency and intensity is around 0.84, which is high. Moreover, my data is not normally distributed, so I must use some non-parametric tests.
I also tried to log-transform the data, but the Shapiro-Wilk Test comes back significant, rejecting the log-normality.
Your measurement design is complicated by the fact that the same items are rated twice. As a consequence, there is probably shared item variance across the two dimensions (frequency vs. intensity) for the items that share the same wording/stem. You could try an approach with a reference item and item specific/method factors for the remaining item pairs.
I would try a model with two correlated "trait" factors (one for frequency and one for intensity) plus method (item-specific) residual factors for each item pair except one reference item pair. In this model, the two trait factors are allowed to correlate and the method (item-specific) factors are allowed to correlate. Correlations between trait and method factors are not allowed. This is known as "multiple-indicator CT-C(M - 1) model" in the context of a multitrait-multimethod (MTMM) design. In your case, the two "traits" are frequency and intensity. The "methods" are the items with the same wording/stem. See
Eid, M., Lischetzke, T., Nußbeck, F. & Trierweiler, L. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod analysis: A multiple indicator CTC(M-1) model. Psychological Methods, 8, 38-60.
Eid, M., Nussbeck, F., Geiser, C., Cole, D., Gollwitzer, M. & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230-253.
Geiser, C., Eid, M., & Nussbeck, F. W. (2008). On the meaning of the latent variables in the CT-C(M–1) model: A comment on Maydeu-Olivares & Coffman (2006). Psychological Methods, 13, 49–57. DOI:10.1037/1082-989X.13.1.49
I think that a bifactor approach would further complicate things as conventional bifactor models come with additional problems plus a conventional bifactor model would not address the item-specificity issue. See
Eid, M., Geiser, C., Koch, T., & Heene, M. (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22(3), 541-562.
https://www.imprs-life.mpg.de/31499/01_eid-et-al-2016.pdf
In terms of non-normality, I would not transform the data. Data transformations come with additional problems and are usually not necessary when you apply robust estimation methods. If you analyze item-level (binary or ordinal) factor indicators, I would use appropriate methods for ordinal data such as WLSMV estimation in Mplus (referred to as DWLS in lavaan). For non-normal continuous indicator variables, I would use robust ML estimation (e.g., the MLR estimator).
- Badges
- Science method