Hi,
I am running a pilot survey before the main national survey to check questionnaire validity and expecting around 40 participants. Can anyone advise me which are the reliability tests I can use with this small sample size?
I have a 44 item likert scaled questionnaire measuring 16 variables. Which tests would be the best ( Cronbach alpha, composite reliability, PCA ,...)?
Dear Matt
1. The concepts:
The concepts and measures of reliability play an important role in many social and applied sciences and are widely applied in psychology and education. In the theory of reliability, there are many forms/categories of reliability measures, each of which has a different concept, meaning and estimates.
2. The reliability measures could be developed for the whole questionnaire or for each dimension/variable of the questionnaire .
3. Reliability consists of several measures:
.Item alpha reliability and split-half reliability assess the internal consistency of the items in a questionnaire – that is, do the items tend to be measuring much the same thing?
.Split-half reliability on SPSS refers to the correlation between scores based on the first half of items you list for inclusion and the second half of the items. This correlation can be adjusted statistically to maintain the original questionnaire length.
.Coefficient alpha is merely the average of all possible split-half reliabilities for the questionnaire and so may be preferred, as it is not dependent on how the items are ordered. Coefficient alpha can be used as a means of shortening a questionnaire while maintaining or improving its internal reliability.
.Inter-rater reliability (here assessed by kappa) is essentially a measure of agreement between the ratings of two different raters. Thus it is particularly useful for assessing codings or ratings by 'experts' of aspects of open-ended data; in other words, the quantification of qualitative data. It involves the extent of exact agreement between raters on their ratings compared with the agreement that would be expected by chance. Note then that it is different from the correlation between raters, which does not require exact agreement to achieve high correlations but merely that the ratings agree relatively for both raters.
Regards,
Zuhair
It may be useful to refer to:
.Landau, S. and Everitt, B. S.(2004). A Handbook of Statistical analyses using SPSS.
.Howitt, D. and Cramer, D.(2008). Introduction to SPSS.
Hello Neethu,
Cronbach alpha on SPSS
Analyze > Scale > Reliability Analysis
While PCA is for dimension reduction , please check this presentation for dr. G. Attar. in the link.
-----------------------------------------
Please also check these links (Q&A)
https://www.researchgate.net/post/What_analsyis_should_be_conducted_for_Correlation_Research_using_SPSS
https://www.researchgate.net/post/How_can_one_help_people_create_simple_scales_from_Likert-scored_items
http://data.library.virginia.edu/using-and-interpreting-cronbachs-alpha/
by Prof. David L Morgan
CFA, or even EFA/PCA is done to examine the constructs' validity and reliability and is related to items, also known as measurement model. Whereas, Regression analysis is to examine interaction of constructs, also known as structural model. (by dr. Ali Farooq)
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size