I have a series of six studies in which all the stimuli and independent variables (conditions in the experiment) are the same but the DV in the experiments are different. For half of the experiments we collected RT as the DV and the other half are eye-tracking studies where the proportion of trials in which participants looked at a distractor stimulus is the DV. I am keen to pool the participant level data into one big mega-analysis where the difference between condition A vs. B could be compared, despite these being different measures. Conceptually the DVs are related (so looking more at stimulus A vs. B leads to slower RT on condition A vs B) but obviously one is a proportion ranging from 0-1 and the other is a RT measure (400-800 ms).
I am wondering how people would approach this analysis? One option I believe would be to standardize (z score) each individual participant's data and then compare responding on condition A to condition B across all participants? RT data is not normally distributed however plus I also believe that this would remove any between-subjects differences. Ideally I would like to compare performance on condition A to condition B, whilst taking into account higher-level factors such as the experiment type (e.g., ET vs RT).
Thoughts appreciated.
Standardization is a methodogical procedure that includes specific steps. Min/max normalization (see formula in literature) is a quite simple way to standardize DVs by excluding outlier observations, making common correlation and unify the units of variables at the same time. I suggest you take a look at the Handbook on constructing Composite Indicators of OECD...
This is just braindizzle level, but you could always just brutally cut down the data to sign level and work with those scores. Like, most extreme case, you just give a +1 to an effect in a given direction for either RT or eye tracking data, -1 otherwise. Cut out the tricky differences in distributions and you safely can test the overall effect and between-group effects of method (I assume it's between-subject data - if everyone did all the studies you could probably do something cleverer with a latent variable model...).
Thank you both, I was not aware of Min/Max Normalization but this does look like a better approach than z scoring. Simply assigning effects as +1/-1 is also a potential way for me to do follow-up between group analyses - thank you!
- Badges
- Science topic