Hi Silburn

One of the easiest way to test for the presence of CMV is to use Harman's single factor. The procedure is simple:

See two publications below for reference.

Mat Roni, 2015 - see the methodology section for guides and citation.

Mat Roni, 2014 (SPSS) - A step-by-step guide to test for CMV.

Cheers.

Saiyidi

Article Introduction to SPSS

Thesis An Analysis of Insider Dysfunctional Behaviours in an Accoun...

Hi Silburn

One of the easiest way to test for the presence of CMV is to use Harman's single factor. The procedure is simple:

See two publications below for reference.

Mat Roni, 2015 - see the methodology section for guides and citation.

Mat Roni, 2014 (SPSS) - A step-by-step guide to test for CMV.

Cheers.

Saiyidi

Article Introduction to SPSS

Thesis An Analysis of Insider Dysfunctional Behaviours in an Accoun...

Hi Saiyidi

Appreciate your prompt feedback and generous contribution on the topic.

Your  Intro to SPSS is an excellently written and well presented piece

Will take a look at the thesis and the particular approach to CMV detection and control/correction.

Silburn

Thanks Silburn.

Appreciate your citation of my work should you find them useful and relevant.

Saiyidi

There is no simple solution to this problem. There are many potential influences on measured variables, some shared across measures and some not. Those shared can inflate correlations and those not shared can attenuate them. With our typical methods, it is very difficult to disentangle method from substantive effects. If you have some idea about the nature of the method variance sources in your study, this will help you decide which approaches make the most sense. For example, if you are concerned about transitory mood effects, separating measures in time can help control for that. However, it cannot control for more stable trait affect, and if that is really the culprit, temporal separation won't help you. You can also use different sources or types of measures (e.g., not all self-report), and that can be helpful, but again it depends on the situation. Method variance can be shared across sources, and alternative sources (e.g., coworker or supervisor ratings) have serious limitations that can produce unshared method variance. A better approach then attempting to distinguish method from trait from item inter-correlations  is to assess potential sources of method variance, such as social desirability. Of course, social desirability is not just a bias, but a personality variable that can have substantive effects. The following chapter goes into more detail on these issues:

Spector, P. E., & Brannick, M. T. (2009). Common method variance or measurement bias? The problem and possible solutions. In D. Buchanan & A. Bryman (Eds.). The Sage Handbook of organizational research methods (pp. 346-362). London: Sage.

Silburn,

I think that your question can only be answered based on your research design and theory. It would be necessary to find out first, which potential sources of method bias may be in your data. Is it common rater bias or measurement context bias, such as using the same measurement instruments for measuring different constructs or measurements taking place at the same point in time or the same location? Obviously, you can only control for these types of measurement bias when there are either several measurement instruments available measuring the same construct or several measurement time points (occasions). Only former research can tell you whether you should expect method bias in your data or not.

Harman’s single test has its obstacles, although this test is often used (sorry, Silburn, I am not so positive about this method). It is unlikely that a single-factor model will fit the data, and there is no useful guideline as to what would be an acceptable percentage of explained variance of a single-factor model. Furthermore, constructs may be highly correlated, as, for example, anxiety and depression. How would you differentiate between explained variance due to method bias and explained variance due to correlated constructs?

You could of course use post hoc statistical detection and correction techniques, such as the correlational marker, confirmatory factor analysis marker, and unmeasured latent method construct approaches (cf. Richardson et al., 2009).

You will find this reference by searching the internet:

Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009): A tale of three perspectives: Examining post hoc statistical techniques for detection and correction of common method variance. Organizational Research Methods, 12, 762–800.

However, you can never be sure whether there is still measurement bias in the data if you did not control it by using several measures for the same construct.

Regards, Karin

Thanks Mohd Saiyidi,

Your Chapters 3 and 4 (Methodology and Results) are quite elaborate in their statistical treatment.  The updated reference to Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. 2012 Review of Psychology article was very helpful.

Appreciate that the selection of the preferred CMB test will depend on a number of influencing factors and will necessarily require some trade-offs.

Harman's certainly has a number of advantages, including its easier operationalisation.  

Was your selection of Harman's based on the particular context and peculiarities of your study,  its  ease of operationalisation (time, effort, costs), accuracy,  integrity etc ?

In the case of my  study,  I am very interested in detecting any shared method variance at the item level (ie manifest variable level)

Your insights would be helpful

Cheers

Silburn

Appreciate your prompt response Paul,

Both your comments / insights and your articles are of immense value.

My understanding is that the Lindell Whitney 2001 correlational approach was replaced by the Williams et al CFA Marker Variable (2003) approach which in turn was itself a competing approach to the Unmeasured Latent Variable approach (various lineage incl Podsakoff & Mackenzie). Richardson et al in 2009 and again in 2015 (Simmering Fuller Richardson et al) have came out strongly in favour of the CFA Measured Marker Variable approach arguing, as you have indicated, that it is impossible / very difficult to disentangle method from substantive effects but also the the unmeasured latent variable approach overly masks substantive variances that are unrelated to a common method bias.Then there is the additional imponderable of separating and distinguishing trait vs method effects

Thanks

Silburn

Hi Karin,

As you quite rightly indicate, the best approach to mitigate CMV in survey research is to craft appropriate research and questionnaire designs that treats with the issue a priori. CMV is facilitated by  same source, same instrument, same time, same environment survey approaches.  Podasakoff et al 2003 groups causes under (1) common rater effects, (2) item characteristics effects, (3) item context effects and (4) measurement context effects. On those bases my study employed manager/employee raters (source differencing) to mitigate the common rater issues;  two waves (temporal differencing) to address the measurement context. The interviewer-administered questionnaire instrument was based on long-standing, theory-based, empirically-tested scales from the literature and was administered by a cadre of trained interviewers using electronic questionnaires on digital tablets' to address,  as much as possible, the item characteristics and item context issues.

The study was aimed at capturing perceptions (of managers, of work, of job, of self-regulation, of behaviour), goal orientations,  as well of output / performance.

The a priori design was deliberately a CMV-mitigation design.

The issue:  how do we test whether the design achieved its design objectives at the item-level as far as CMV is concerned.

Against the backdrop of the various caveats on trait vs method, shared vs non-shared variance, inflated vs deflated effects and drawing on the more recent literature,  the preferred litmus test that is appearing seems to be the CFA measured marker variable approach (also recommended by Richardson et al 2009 and Simmering Fuller Richardson 2015) as the most promising for controlling CMV. 

However, before engaging a full-blown CFA measured marker variable based correction protocol at the item level, the guiding thresholds  would need to be less inconsistent; for example, one researcher asserts that if the differences in standardised regression weights between items (generated with marker and without marker) are less than 0.200 units then CMV is absent.  It this a correct position ?

Silburn

It is hard to say that it is a correct position, but it is a position that some reviewers might accept. Another approach is to look at statistical significance in regression weights to see if they go down significantly. Yet a third is if the regression weight maintains significance. All have limitations.

A different approach is to include measures of method variance sources that you expect, and model their effects. Examples of this approach are:

Williams, L. J., & Anderson, S. E. (1994). An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79(3), 323-331. doi: 10.1037/0021-9010.79.3.323

Williams, L. J., Gavin, M. B., & Williams, M. L. (1996). Measurement and nonmeasurement processes with negative affectivity and employee attitudes. Journal of Applied Psychology, 81(1), 88-101. doi: 10.1037/0021-9010.81.1.88

Silburn, I generally don't like rules-of-thumb cut-off criteria, because they are not population values but rather sample-related values. In a different sample, .20 might mean something else as in your study.Therefore the value of .20 is quite arbitrary.

Thinking about your study I am not quite sure why you actually want to investigate CMV at the item level. You already control for rater effects and occasion-specific effects, which is a lot and not often found in empirical research (at least in psychology). What kind of method effects do you expect at the item level? Are your concerns related to former research which pointed to potential problems of the items, or do you need to react to reviewers' concerns?

If you still need to test CMV at the item level, Paul provided some valuable suggestions. 

Regards, Karin

Thanks for the insights Paul / Karin,

Have now tested two approaches using SEM at the manifest variable level: 

a. Common Latent Factor:  CMV test result was measured at 7.8% but in effect  this number would have masked all influences of shared variances under one number.

b.  the CFA Marker Variable technique  which employs Chi-square comparisons tests  on several marker-variable  applications in SEM models  (Rafferty & Griffin 2004, 2006;  Williams, Hartman & Cavazotte 2010).  Tests yielded method variance results of 1%

Silburn