I have completed pilot test of a research project. The research model consists of constructs with items (in 1-5 likert scale). I need to run a suitable statistical test (reliability analysis) before the main study.
Could you please suggests some suitable statistical tests?
Regards,
Gobinda
Hi!
In statistical softwares such as SPSS and SAS, cronbach's alpha and the KMO-MSA measures are reported. These may be a good start to see if your questionnaires are "reliable" or "consistent"
Hi!
In statistical softwares such as SPSS and SAS, cronbach's alpha and the KMO-MSA measures are reported. These may be a good start to see if your questionnaires are "reliable" or "consistent"
Dear Gobinda,
For a pilot study with a small sample, if you are trying to establish the reliability of a collection of Likert response items to describe the same construct, I would recommend starting with a Principal Component Analysis rather than a reliability analysis. The things to look for are the size of the first eigenvalue and the loadings on the first component.
There are more details in my guide: https://www.researchgate.net/publication/280936182_Advice_on_Reliability_Analysis_with_Small_Samples
Technical Report Advice on Reliability Analysis with Small Samples
Hi!
To check the reliability of the data, in the SPSS software you can obtain Cronbach's alpha value, KMO and Bartlett's test of sphericity. Also you can go for Principal Component Analysis.
Hi,
Each test has its own purpose. If you want to check the consistency of your data collection tool, then go for reliability test. In order to reduce the number of variables under a particular construct, use principal component analysis (factor analysis).
Thanks you all for your valuable feedback. In face I did some research about suitable analysis tests for pilot study data. Many research papers prefer to perform only descriptive statistics (Bunn et al. 1998; Bauhofer et al.2001; Carfoot et al. 2002) and "A pilot study is not a hypothesis testing study. Therefore, no inferential statistical tests should be proposed in a pilot study protocol. With no inferential statistical tests, a pilot study will not provide p-values."
Source: Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of psychiatric research, 45(5), 626-629.
For the stated question above, I have already validated scales from previous studies. Hence, all cronbach's alpha values are known. Now I run reliability analysis on each construct, alpha and found alpha values are less than values stated in published journals.
Example; my model has 7 construct and 35 items. I have collected 50 pilot responses and did reliability analysis and found overall alpha >0.8 ( good). But when I run reliability test on each construct, I found alpha values in many cases about 0.07 (which are lower than stated values in reputed journals). Is there any reason for such lower values? Is it because of smaller sample size?
Thanks Peter for sharing the research brief about analysis with small samples. It was really helpful.
I also request your advice on the above stated findings.
Regards,
Gobinda
Hi,
The Cronbach's alpha is less, when there is less or no inter-correlation between the items. If the items under a particular construct are correlated, then alpha will be high. Make sure the items have consistent opinion.
Thanks for the information Peter. wanted to know if factor analysis can be done in the pilot study?
- Badges
- Science topic