Say my experiment with repeated measures design is concerning the effect of lighting on memory, and I measure memory by giving participants a random list of 50 words to memorize. Would my experiment be flawed if I use a different list of 50 words for participants to memorize after each IV level, so as to avoid practice effect?
Not necessarily. There are various approaches for confining the problem of measurement invariance: you can test for construct stability / measurement invariance with special tests, use latent modelling (e.g. in a cross-lagged panel design), or (typical for competence testing) include anchor items that bridge from one measurement to the other and item-response-modelling has a similar effect. However, I would advise you not to handle the measurement as invariant without checking.
Search for a memory test with at least one prallel version, which have been shown to have the same item difficulty etc. For example:
downloads.hindawi.com/journals/bn/2004/940191.pdf
There could be "learning effects" with your dependent measure if for example the practice of trying to memorize words in the first round helps your participants develop better strategies for memorizing subsequent (albeit different) words later on. You can check for an order effect, but as long as you counterbalance the independent variable and either counterbalance or randomize the word-lists that people will have to memorize, I think your experimental design would be okay. That's my two cents, anyway.
- Badges
- Science topic
- Similar topics
- Psychometrics
- Measurement