Greeting everyone,
I wanted to ask for elucidations regarding EMA and the statistics behind it.
For a background, I was thinking of developing a study in which 2 groups of people will be assessed by means of EMA for 4 days in three different moments, aka week 1, week 3 and week 5. The data collected should be related to emotional responses and will be assessed by self-rated questions on EMA multiple times a day. The sample size should be round 40-50, divided in two groups
I am not familiar with EMA studies, for what I've seen there're many methodological issues regarding how to analyze the datas.
For one, in a setting such as the one exposed, given the multiple levels of measures (3 times for two groups), could aggregating the data for the three different times (ie means of emotional reactions) and then analyze by mean of a repeated measure ANOVA be a good solution? Or is it absolutely necessary to employ a multilevel model? I've seen studies employing differently both t-test mean ratings or multilevel model approaches, but I'm not really familiar with multilevel models and nearly all have been employed in more simple settings (ie single time or two times but single group).
On a similar note, in accounting for individual variability, I've seen using both ICC or mean squared for successive differences (MSSD) as approaches used. In particular, MSSD seems similar to aggregating responses as an approach. Should I use both or is one superior to the other? And why so?
I hope I've been able to expose my doubts in a comprehensive manner, thank you beforehand.
There are many different methods for EMA, but if you are just considering these two. Then the overall tests should be equivalent if sphericity is met, you have complete balance and no time-varying covariates. However, I'm not sure that you'll realistically get balance because of missing data and EMA without time varying covariates seems like an overly complex design for just looking at the overall averages (i.e. why design such a complex sampling scheme if you then ignore most of the complexity?).
A multilevel, multinomial modeling approach would be a good choice for EMA data: Article Multinomial analysis of behavior: statistical methods
(Note that temporal autocorrelation might be a consideration if there is a short interval of time between observations.)
Thom Baguley not really sure if I've fully understood your insight, if your doubt concerns the reason why I would do an assessment in 3 different weeks is because the condition in which I would make the assessment are different, aka different treatment (so t0, t1, t2), and I'm interested particularly in assessing the average differences in emotional responses in the three different conditions.
For what regards the balance I guess you're meaning the difficulty in realistically have the same number of responses in the individual, that is a often encountered problem in EMA setting, that's why I was concerned about if simply averaging the response values and then impelling an ANOVA would actually be feasible (as in this arcticle, where they impelled a t-test to assess the difference in average response and average variability in the 2 different condition: doi: 10.1016/j.nicl.2019.102032) and to what extent, as if there's a limit to it (dunno, maybe limiting the acceptable data to 80% response rate).
If you've got more intel or suggestion regarding the design may I ask you where could I get some methodological background?
Thank you beforehand
If there's missing outcome data then using the average isn't ideal (as is losing information by dropping incomplete cases or < 80% incomplete cases). The multilevel model should be less biased and may have greater statistical power. The average analysis basically overweights cells with missing data and hence overweights noisy responses.
There may be other good reasons for your design but in most EMA designs there are time-varying covariates that will be ignored or not dealt with adequately if you just average. Here I think time of day is the obvious factor not handler properly if you average t0, t1 etc. So not special insight here - just the intuition that an analysis that ignores the structure/timing of data collection might miss something important.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size