As you point out there are different ways to treat the multiple measurements. One way that depends on how you take the multiple measurements is called longitudinal models. You can find a set of notes written by Prof Marie Davidian, a real expert in the field at:

https://www.stat.ncsu.edu/people/davidian/courses/st790/notes.html

Start with repeated measures and then go as far as you want into the subject. This is really useful stuff for design and analysis of experiments. Best wishes, David Booth

As you point out there are different ways to treat the multiple measurements. One way that depends on how you take the multiple measurements is called longitudinal models. You can find a set of notes written by Prof Marie Davidian, a real expert in the field at:

https://www.stat.ncsu.edu/people/davidian/courses/st790/notes.html

Start with repeated measures and then go as far as you want into the subject. This is really useful stuff for design and analysis of experiments. Best wishes, David Booth

As to how this can be put into power analyses:

A useful tool for power-analyses is the free G*Power software. You can just play around with different numbers of measurements to get a feeling for their influence on the test's power.

You can find the download, the manual, a tutorial and scientific background under: http://www.gpower.hhu.de/

Hi Jim,

Multiple measurements will generally increase power. The influence of outliers will be diminished, especially for highly variable measures like response time.

For some measures, (particularly continuous measures) you run the risk of oversampling. But, it seems like you primarily use discrete measures.

The other considerations are methodological. You have to weigh the trade off between increased power and order/practice effects, which have the potential to adversely influence your validity.

The only time that you would treat the repeated measurements individually would be if you were interested in change over time. In which case, you would treat trial order as a within-subjects variable. Otherwise, you will artificially inflate your power. I have always averaged across repeated measurements, but it looks like David suggested some intriguing, more sophisticated approaches to this issue.

Best of luck!

Brandon

Thanks to David Eugene Booth, Martin Westhoven and Brandon Thomas for your responses, these are all really helpful.

I guess the crux of my question is how you can factor in the potential increase in power obtained by taking multiple measurements (and then averaging these multiple measurements) when carrying out an a priori power analysis to determine the sample size required for an experiment.

For example, consider I am carrying out a simple experiment with a single, two-level between-subjects factor, and I plan on using an independent t-test to assess the difference in responses between two groups. I anticipate finding an effect size of Cohen's d = 0.5. How can I assess what size sample I require if I plan on taking 3 measurements of the same DV for each participant and averaging across these 3, as opposed to if I were taking only one measurement from each participant.

Martin Westhoven , GPower is a great piece of software. If I use it to assess the required sample size for an independent t-test, I can't see whether I can account for the number of measurements I might take from each participant. Perhaps I am looking for something that doesn't make sense or is not possible to factor into power analysis?

Thanks again for the responses.

Jim

If I am not entirelly mistaken, you're not really looking at independent measurements anymore, when using repeated measurements. If you average across the measurements, you effectively drop the information about how many measurements were taken. Also, a bias could be introduced to the averaged value through the order/practice effects Brandon Thomas already mentioned.

So while averaging allows you to still use independent t-tests, you have to accept some drawbacks.

You could try an ANOVA for analysis, though. Repeated measurements ANOVAs are built for this kind of mixed design. Last but not least, G*Power factors in the "number of measurements"-parameter for RM ANOVAs / MANOVAs, so you can compute the required sample size with relative ease.

The power computations for RM ANOVAs would also be my starting point to a deeper understanding of the mathematical background of repeated measurements effects on required sample sizes.

Hi Jim Uttley , I realize this response is coming maybe a bit later than you may have wanted, however I hope it can still be helpful.

For multiple measurements per person, you can calculate power using GPower just like you would for an OLS regression or non-repeated measures ANOVA (using GPower3, assuming you are familiar with that software) but adjust sample size for dependence between measurements. This method is appropriate for 2 level cross-sectional & repeated-measures models, which to me sounds like it fits your case. To do this, you would calculate what's called "Neffective." Neffective = N / (1+[n-1]*r), where N = total number of data points, n = group size, and r is the intraclass correlation (ICC) between measurements(or timepoints). In your case it sounds like group size corresponds to number of people in your study. In case you are unfamiliar with ICC, here's a web resource that's fairly well-written: http://www.theanalysisfactor.com/the-intraclass-correlation-coefficient-in-mixed-models/. And here's a link on how to calculate it in SPSS (Article Intraclass correlations: Uses in assessing rater reliability

Its interpretation is such that the value of the ICC corresponds to the % of variance between groups (between your 2nd level, or person level), whereas whatever is your finest level of measurement (trial, time, etc.) is the leftover percentage of variance to be explained. To make this more concrete, for repeated measures design (your case), assuming a multilevel model approach where trial is at level 1 and person is at level 2, an ICC of .41 would indicate that 59% of the variance in the outcome is within-people and 41% of the variance in reading achievement is between-people.

Once you calculate Neffective, you would simply use this as the sample size in GPower3 when calculating sample size in order to provide an accurate assessment of power. If you use Neffective, the appropriate citation would be: Kenny, D. A., Kashy, D. A., Cook, W. L., & Simpson, J. A. (2006). Dyadic data analysis (Methodology in the social sciences). New York, NY: Guilford. Along with the citation for GPower3. 

Another option if you go the Multilevel Modeling route that I'm aware of is using PINT (power in two level designs) software. It can be accessed via this link: https://www.stats.ox.ac.uk/~snijders/multilevel.htm#progPINT. Unfortunately, this software cannot handle models greater than 2 levels, but might be appropriate for the analyses you would like to perform. Its a little more complex than Neffective. There is more information on Snijders website.

Finally, conceptually, I would almost always want my research design to match the model used - so I would recommend using each of your measurements, rather than averaging over them. This will not only increase your power but also give you the most accurate estimates of parameters in your statistical model, since your model will exactly match your experimental design. And now you (hopefully, if I was clear enough) have a method to account for the interdependency of measures, and still get an accurate estimation of statistical power.