I am conducting a perceptual judgment experiment. It is a between-subjects design: feedback group vs non-feedback group in which each participant consists of 200 trials.
I know how to use G*Power to calculate the sample size, as attached. However, I wonder why we don't take the number of trials (observations) into consideration when determining the sample size. I think the more trials, the better sensitivity, and the fewer participants needed.
Hello Cuizhen,
The answer depends on whether your intention is to:
1. Use a summated or averaged value of scores across trials as the DV of interest; or
2. Use trials as a within-subjects variable (or treat trial scores as nested within participant, as in a multi-level model).
For #1, Individual differences across the aggregate score still are the basis for the error term in parametric tests, so with one exception (estimate of a participant's true score) there's no real sensitivity gained by having the aggregate DV value be derived from multiple trials.
For #2, in the mixed model (one-between IV, one-within IV), then for the main effect of treatment, again there is no change as the "scores" for comparing groups are the summation of trial scores.
For #2, in multi-level model, the number of trials will be important, up to a point (e.g., you want a sufficient number of trials to be able to estimate case-score slopes with some confidence). However, sample size determination might better be accomplished via simulation methods (see https://www.bristol.ac.uk/cmm/learning/multilevel-models/samples.html).
With hundreds of trials, there can be a risk of fatigue or boredom affecting performance, which actually would serve to decrease accuracy of any aggregate estimates of participant performance.
Good luck with your work.
Cuizhen Liu -
You said you were interested in a "between-subjects design." That sounds like more trials would be used to increase accuracy for an individual, but your main interest is between individuals, as explained by David above. You could think of an errors-in-variables model where the trials are to reduce measurement error. But David has a good point here about fatigue. Anyway, yes you want an accurate account for each participant, but the main concern is enough participants, if I understand your question.
I haven't used G*Power in a very long time, and I don't know what is available to you there as far as a multi-level model.
Cheers - Jim
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers