For estimating test-retest reliability through ICC (intraclass correlation coefficient) two models are provided by SPSS i.e. one way and two way models (random and fixed).
Which one should be used if I collected data from 50 respondents at time 1 and recollected same data from same respondents at time 2. I want to see stability in their responses on questions asked.
From a paper by Kennth O Mcgraw titled “FORMING INFERENCES ABOUT SOME INTRACLASS CORRELATION COEFFICIENTS”, I arrived at a decision that one way model should be used if respondents are the only source of variance and columns variation (time in this case- which is asumed to have no influence on respondents in case of test-retest reliability) is irrelevant.
On this basis should I use one-way model. Please suggest ……..also tell whether consistency or absolute agreement should be checked.
Look up the link below. It may give you some perspective on your question.
http://www.ncbi.nlm.nih.gov/pubmed/15705040
Jagdish, below is a reference that might be helpful
Shrout, P., & Fleiss, J. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86 (2), 420-428 DOI: 10.1037/0033-2909.86.2.420
There is also an interpretation given by the link, below:
I think you need alpha and not ICC. The ICC for reliability is if you had coders interpret data from participants. When data is from the participants and is repeated, you need to consider alpha coefficient.
http://neoacademic.com/2011/11/16/computing-intraclass-correlations-icc-as-estimates-of-interrater-reliability-in-spss/
i recommend that you look up at the link below and this may help you get some insight.
http://www.ncbi.nlm.nih.gov/pubmed/15705040
Dear Joan Vaccaro,
Thanks for answering my question. I have some doubts about alpha as it is a measure of internal consistency (items measuring a particular construct should correlate with each other), in my questions unidimensionality is not required (no construct concept is involved). for eg, in the following two questions, no internal consistency is desired.
Q1: Which factor(s) in your opinion determines exchange rates? (Rate from 7 to 1)7 means most important and 1 means least important
Q2: Following your trading strategies how you would rate your success rate in foreign exchange trading? (Rate from 7 to 1)7 means most important and 1 means least important
Factors:
Someone, please suggest how can I check the reliability of the above two questions.
use only one way test in spss for comparation the mean (average) of data before and after
actually, for stability test from data for the same responden just make the correlation analysis (pearson product moment) from total first data summary to second total data
if significant for 1% alpha, very stabil, for 5% alpha stabil, for 10% alpha moderate stabil,
The stated goal is to test temporal stability of respondent choices, and hence the 1-way based ANOVA for ICC is one such option. It appears that you have 4 domains within each of two questions (total: 8 "items") to assess, each "item" comprising one analysis model. Your data however, as a rating scale from 1-7, may be marginal for use with ICC which is designed for (at least) interval level data. Most behavioral researchers assume that an ordinal scale of at least 7 categories can behave like interval level data, but the distribution of responses should be checked in order to verify that all 7 levels are being used.
A very simple and intuitive explanation on the correct choice of the ICC is reported in this paper
Article A Guideline of Selecting and Reporting Intraclass Correlatio...
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS