I have a data set from 10 subjects with three measures that are repeated in 5 trials and 3 sessions; additionally, the values of the 5 trials are averaged for each session. Thus, the data set is structured using the following columns:
Subject (10), Axis (4; cardinal and resultant), Session (3), V1_avg (value), V2_avg (value), V3_avg (value), V1.1 (value), V1.2 (value), V1.3 (value), V1.4 (value), V1.5 (value),..., V3.5(value)
I am not confident that the data are framed correctly. Should they be stacked? Currently, there are 21 columns and 120 rows.
I know how to calculate the ICC using SPSS, but I am learning R and want to know how to calculate reliability using this. What is my best approach. I have tried a few different approaches and packages with little success. The examples I can find don't seem to help. This should be a very simple task!
Thank you!
uhmn. I don't understand completely your situation but maybe you'll looking for ICC function for psych package http://personality-project.org/r/html/ICC.html
ICC coefficients index consistency or reliability in some measure across repeated testings, with reliable measures reflected by high ICC values. ICC is supported by the R software package (using the icc command with packages psy, psych or irr)
William, I cannot figure out how did you get 120 rows. That means 12 rows per subject. From your previous description of your data I arrive to 10 rows with 21 columns each.
I think 10 cases is a short sample for reliability analysis. Besides, you may have two reliability problems: scale internal consistency and time consistency.
To solve your first problem, Omega looks fine (http://personality-project.org/r/html/omega.html). Here I'd enter V1.1 to V1.5 as target variables. A high Omega (hierarchical) value is a good index of internal consistency (your 5 trials can be summed up and your V1_avg is supported). You may repeat this to ensure everything is OK in your 3 sessions.
Your second problem has to do with stability of measures over time. In this case, I think it is enough to compare V1_avg, V2_avg, and V3_avg, using a proper criterion. if you expect stability of scores ICC should be high (i guess ICC3). If you are expecting unstability, mean differences should appear, but pearson r should be high.
Thank you for the responses. In the short-term, I ultimately resolved my challenges in R--albeit more clumsily than I would like. The 120 rows is erroneous. Stacked, there are 10 subjects repeated for 3 sessions of 5 trials (150 rows); wide, there are 10 rows and 21 columns as defined above. The 120 included 30 rows (10 subjects by 3 sessions) for each axis (4). I am able now to compute the ICCs using R {irr}.
I am still not satisfied with the result. The three variables are a reduction of an analysis that returns a data set that has 11 columns and 6250 rows. I am most interested in whether this "cloud" of essentally 3-dimensional data differs from subsequent "clouds". The 3 variables are an application of PCA. I ran the ICCs on each, but it would seem that I am still missing some of the full picture. Ideally, I would like some correlation based on the entirety of the data--similar to digital volume correlation (for lack of a better comparision). The math involved is a stretch for me. It seems to me that PCA is still involved, but I see this as a need for an ICC calculation based on the linear combination of my variables, similar to doubly MANOVA vs. repeated measures MANOVA.
I am not sure if this makes sense to anyone but me, but, if it resonates with anyone, I welcome the input.
thanks!
Jeff
(I agree 10 cases are not many. It is a prelimary study, and the data sets are quite large--150 files with 11*6250 cells for three axes and the Euclidean resultant, i.e., 600 files.)
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS