Hello Frida,

I think I understand your concerns. Please excuse info where you may already be informed -- just trying to cover a few areas.

If you have some variation in item selection within a CFA for separate groups (your subgroups), then you may have some level of invariance. Which items and what level of invariance need to be tested.

Rather than comparing the groups in isolation to start (as you seem to intimate with your CFAs), I suggest you are best to conduct a combined CFA and then test for invariance across groups. From these results you can determine the level of invariance (metric, scalar, and strict) and which items are invariant. See Fan and Sivo (2009) and Cheung and Resnvold (2002) for test statistics.

Note that unequal sample sizes tend to reduce invariance test statistics, therefore, invariance tests are more likely to fail to detect noninvariance in such situations (Chen, 2007, p. 499). Chen (2007) has suggested adjustments to fit criteria for unequal group size; also see Yoon and Lai (2018).

To what I think is your main point, it is becoming common practice to accept some violation of measurement invariance (i.e., to claim and use partially invariant constructs). Using a partially invariant construct to compare across groups has been conducted when at least half of the items are invariant (Steenkamp & Baumgartner, 1998; Vandenberg & Lance, 2000). However, such heuristics are not the result of empirical studies and may produce biased results. A good read in general is Putnick and Brrnstein (2016). Chen (Chen, 2008) presents a good discussion on the level of bias in comparing means, with some general guidelines.

Sourcing the non-invariant items is a little contentious. Some suggest releasing one item at a time, starting with the highest modification index (Yoon & Kim, 2014), while this approach has been viewed as problematic when noninvariant items is high (Yoon & Millsap, 2007). Others have noted that the scaling of the factor is important, with suggestions that using a fixed factor method over the marker variable method is preferred (Little, 2013).

On the mechanics of dealing with non-invariant items, it is generally recommended to release the noninvariant parameters in your model to account for partial invariance; Guenole and Brown (2014) present the rationale for this well.

I hope this was what you were looking for.

With best wishes in your research,

Tim

REFERENCES:

Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. doi:10.1080/10705510701301834

Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95(5), 1005. doi:10.1037/a0013193

Cheung, G., & Rensvold, R. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255.

Fan, X., & Sivo, S. A. (2009). Using Goodness-of-Fit Indexes in Assessing Mean Structure Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 54-69. doi:10.1080/10705510802561311

Guenole, N., & Brown, A. (2014). The consequences of ignoring measurement invariance for path coefficients in structural equation models. Frontiers in Psychology, 5, 15. doi:10.3389/fpsyg.2014.00980

Little, T. D. (2013). Longitudinal structural equation modeling. New York, NY: The Guilford Press.

Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71-90. doi:10.1016/j.dr.2016.06.004

Steenkamp, J. B. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross‐national consumer research. Journal of Consumer Research, 25(1), 78-107. doi:10.1086/209528

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. doi:10.1177/109442810031002

Yoon, M., & Kim, E. (2014). A comparison of sequential and nonsequential specification searches in testing factorial invariance. Behavior research methods, 46(4), 1199-1206. doi:10.3758/s13428-013-0430-2

Yoon, M., & Lai, M. H. C. (2018). Testing Factorial Invariance With Unbalanced Samples. Structural Equation Modeling: A Multidisciplinary Journal, 25(2), 201-213. doi:10.1080/10705511.2017.1387859

Yoon, M., & Millsap, R. E. (2007). Detecting violations of factorial invariance using data-based specification searches: a Monte Carlo study. Structural Equation Modeling, 14(3), 435-463.

Another suggestion is comparative network analysis as a formative approach to items where these relate to each other and complement each other because the invariance of the dynamics of certain indicators of the group that has no invariance.

Thank you very much Tim for your detailed and valuable explanation. Thank you Cristian for your valuable information. Will take all your suggestions into my work.