Tahir,

1. Construct a PCA model using all the data with vector Vexpt for each sample.

2. For each sample calculate the vector  Vmodel using only N of the PCA components.

3. Calculate the length L of the residue vector (Vexpt-Vmodel) for each sample.

4. Outliers correspond to large values of L

You will need to experiment with what values of N and L gives you the clearest detection of outliers.

Regards

Paul

What program are you using, Tahir? As Florian suggests, most SEM programs will produce outlier stats (e.g., Mahalanobis distance, Cook's D) for each observation on a latent construct, along with their p-value. Often the p-value is judged somewhat more conservatively in the instance of Mahalanobis distance for SEM. Kline (2011) suggests p

I'm using AMOS and SPSS....  So I guess AMOS will be helpful... 

AMOS does provide info. From memory, you tick a box for testing 'normality and outliers' under Output tab, within Analysis Properties. 

You can make a boxplot of the Mahalanobis distance

I am not sure if this is applicable on your data, but density based clustering can be employed to detect clusters. E.g. the DBSCAN algorithm can give you the number of clusters and also the cluster labels for each point, while also accounting for outliers. DBSCAN basically returns a class labeled  "0"  corresponding to your outlier points. Other more specialized density-based methods for outlier identification have been proposed such as the OPTICS (Ordering Points To Identify the Clustering Structure).  See for instance the link:

http://www.dbs.informatik.uni-muenchen.de/Forschung/KDD/Clustering/

So Tim and Jose... I ran the data in AMOS and using Mahalanobis distance I could find 100 cases that have scores of p1 below 0.05. But on observing each case I find them normal and engaged responses.  So what shall I do?  Shall I keep them or remove them?  Can u cite any refrences please. 

If the raw data is a set os qualitative variables you can apply the multiple correspondence analysis and look the coordinates of the respondents on the first factorial axis. You can apply the R packge CA.

Hello Tahir. Ok, 100 cases is a substantial portion of your survey detected as outliers. A few issues to consider. First, I would take a strong look at your model for standardized residuals (i.e., look for values over 3) and modification indices. Do any of these highlight issues? Common wording of questions (common method issues), poor items, ordering that biases answers. If you have sound reasoning, perhaps freeing some of these items might be possible … but you need to back this up with sound reasoning/theory. This may or may not help with some of the outliers.

Alternatively, you could drop the 111 (which would be unfortunate). However, you would like to have sound reasoning as to why these cases are not part of your intended (representative) group being analysed; a case which you seem to discount in your previous comments.

Probably where you need to be heading (assuming you have a sound theoretical model … and measures) is to use a robust ML estimator. Most social science latent variables are not really normally distributed – they tend to have long tails. Although large sample sizes and ML can overcome some non-normality, it has been well established that ML assumes normality. Robust ML estimators are used for non-normally distributed data. Robust estimators used in SEM ‘largely’ overcome the outlier problem. A reference for this is:

Yuan, K.-H., & Zhong, X. (2013). Robustness of Fit Indices to Outliers and Leverage Observations in Structural Equation Modeling. Psychological Methods, 18(2), 121-136. doi: 10.1037/a0031604

 As I remember, AMOS only has ADF (Asymptotically distribution-free) as an estimator for this purpose. I have found this less than satisfactory in AMOS; it requires large data sets and does not permit missing values on ANY variable. You may have to make the jump to programs like Mplus or R. I tend to use Mplus where it is fairly straight forward to use robust ML and missing data is imputed (using FIML).

Best of luck with your research.

Thanks again Tim,

What I have done now is generated box-plots and Skewness and Kurtosis results everything seems to be OK from there (in the limits of +-2). Then I stumbled upon this link and found the author has found above 35 cases out of 250 cases to be outliers using Mahalanobis  D but has not removed any. Plus most of my factors have been taken from standardized scales.

http://scholar.lib.vt.edu/theses/available/etd-040999-201108/unrestricted/CHP6.PDF

Hi Tahir. Sounds encouraging. Presumably you are describing stats on the constructs, as opposed to the item measures. This would 'suggest' that the outliers 'may' not be extreme. However, keep in mind, that Mahal. is a multivariate test for outlers - more than looking at individual measures/constructs. However, how much impact it has is the key issue (another stat to consider is Leverage). 

Maximum Likelihood assumes multivariate normality, but is somewhat robust. Unfortunately, there are no hard and fast rules, due to the various inputs into ML (means, variance, covariance). A more modern approach put forward by some researchers in the stats area, suggest comparing the parameters when using robust ML vs normal ML (mainly SE and chi-square; regression parameters tend not to change much). If there is no significant difference, than non-normality is not a problem (standard errors are the main issue). The following may be helpful to you:

Muthén, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non‐normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38(2), 171-189. doi: 10.1111/j.2044-8317.1985.tb00832.x

ML is fairly robust to violations of normality assumptions:

Diamantopoulos, A., Siguaw, J. & Siguaw, J. A. (2000). Introducing LISREL: A guide for the uninitiated, Sage Publications.

Bollen, K. A. (1989) Structural equations with latent variables, Wiley New York.

Reinartz, W., Haenlein, M. & Henseler, J. (2009). An empirical comparison of the efficacy of covariance-based and variance-based SEM. International Journal of Research in Marketing, 26, 332-344.

Bootstrapping can help get around the issue:

Preacher, K. J. & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, & Computers, 36, p. 717.

Large sample sizes help:

Hair, J. F., Black, W. C., Babin, B. J. & Anderson, R. E. (2010) Multivariate Data Analysis: A global perspective Upper Saddle River, NJ, Pearson Education Inc.

However, severe non-normal data (multivariate!) would likely require an estimator more suited to the data (see previous post).

I hope this is helpful, Tahir. Pick your interest and see if any of these references help you back your case.  Wishing you well in your research.

i agree with Tim

I would also look into the "Forward Search for Data Analysis" method, it is designed to detect multivariate outliers. Basically you focus on a specific statistic, e.g. Mahal. distance, you compute it on a robust starting set of observations (obtained with from e.g. a robust clustering algorithm) and then you add one observation at-a-time and watch the evolution of the statistic values.

http://ipsc.jrc.ec.europa.eu/?id=732

Use residual analysis to detect outliers

If using Stata, you can read http://www.stata-journal.com/article.html?article=st0197

If SPSS http://www-01.ibm.com/support/docview.wss?uid=swg21480128

Have a look at this video. It is very useful.