My question is about whether there is an established procedure for investigating interactions in EEG analysis.
Some papers seem to correct for multiple comparisons and others do not, so I am wondering if there is an agreed perspective as to when correction is needed. To use a relatively simple example, if I have a 3 way interaction between Factor 1 (three levels), Hemisphere (Left, Right) and Region (Anterior, Central, Posterior), I would investigate this by doing a single-comparison between the levels of Factor 1 in each combination of the Hemisphere*Region interaction.
But I have seen different approaches used to achieve this: One approach would be to take a subset of each site (i.e., Left-Anterior, Right-Central etc.) and do a one-way anova to investigate the effect of Factor 1 at each of these sites, then run (uncorrected) contrasts to investigate which levels of Factor 1 differ. The argument here is that the uncorrected follow-ups are licensed by effects/interactions in the preceding interactions.
Alternatively, a much stricter approach is to run corrected t-tests (e.g., Bonferroni) at every possible combination of Factor1*Region*Hemisphere (which by most methods in R/SPSS includes correcting for comparisons I am not interested in (e.g., Level 1-Anterior-Left vs. Level3-Posterior-Right and thus produces very strict p-values). So I am interested in what EEG experts would do and precisely when (and to which single comparisons) they would apply a correction procedure – if any?
It may seem like a simple question, but it is one for which there seems to be very mixed approaches across the many papers I have read.
Hi Ryan,
is there an á priori reason to assume a hemispheric difference or specific ROI? Right now, it's not clear to me why you want to make your analysis much more complicated. If you don't include the factors hemisphere and ROI, you simply end up with a one-way anova with 3 levels, which you can compute at each electrode and correct for multiple comparisons.
Could you maybe explain in a bit more detail, what you are looking for?
Best,
Julian
Many thanks for your response, Julian.
Based on previous literature, I think there is a-priori assumption for hemispheric difference/ specific ROI: I am looking at particpants’ response to syntactic violations and thus expect a P600 (maximal at centro-parietal regions [i.e., Cz, Pz)] and LAN (negativity maximal at Left-Central and Left-Anterior sites).
The reason I have outlined a 3x3x2 Anova above is that previous papers interested in these effects seem to have used this approach (some correcting for multiple comparisons and others not). Would you do it a different way? And if so do you have any opinion on correcting for multiple comparisons? I'm still learning about EEG analysis so I'm open to suggestions.
Hi Ryan,
I see, but are you expecting an interaction in your 3x2x2 (Condition x ROI x Hemisphere) ANOVA? Would it be possible to eliminate one factor by computing for example the hemispheric difference (i.e. take two homologue electrodes and compute left-right). Then you end up with a 3x2 (Condition x ROI) ANOVA, which you can then compute for each timepoint and subsequently correct for multiple comparisons analoguous to a serial t-test: Here, you'd need to loop through your data for each timepoint and then use a time stability criterion (e.g. only significant if stable for x ms) to correct for multiple comparisons. This approach is described in Picton et al., 2000 and Guthrie and Buchwald, 1991. Please note however that this is far less conservative than e.g. cluster-based correction or Bonferroni correction.
Alternatively, if you in fact have strong hypothesis on the interval (e.g. 500-600 ms) and ROIs, it is more straight forward to just average over intervals and ROIs and use this data in a full-factorial 3x2x2 ANOVA. Depending on the Outcome, you can either reduce the ANOVA (e.g. if you don't find a 3x2x2 interaction, you need to take this interaction out of your model) or split the data by one factor and compute 2 separate 3x2 follow-up ANOVAs which you can then explore using t-Tests or specific Post-Hoc tests (e.g. Tukey HSD).
I have no idea how the other papers you mention are doing this, but I generally try to reduce the complexity of the comparisons as much as possible (optimally down to a t-Test). In any way, you need to correct for multiple comparisons and adapt the model you test.
Good Luck
Julian
http://onlinelibrary.wiley.com/doi/10.1111/j.1469-8986.1991.tb00417.x/abstract
Article Guidelines for Using Human Event-Related Potentials to Study...
To summarize:
(1) of course you must consider protection against proliferation of alpha error when doing multiple comparsions, also in EEG
(2) But nobody hinders you to select as few comparisons as necessary to proof a certain hypothesis. Make a simple model including your effect of interest, then control for the famiiliywise error rate (FWER). 1
(3) Use a correction algorithm which is not so much conservative as Bonferroni, there are better options, e.g. the "False Discovery Rate" according to Benjamini & Hochberg (about 1991 or something).
You might want to look at the cluster permutation statistics present in field trip. The paper by
Article Nonparametric statistical testing of EEG- and MEG-data
"which by most methods in R/SPSS includes correcting for comparisons I am not interested in"
This case it is a viable method to do the Bonferroni "manually". First you should decide how many comparison are you interested in (e.g. only the same channels in a given comparison between faktor1 and faktor2). Then you do a simple t-test, and you set your p-value accordingly. If you are interested, let's say, in 5 comparisons, then you set your criteria to p=0.05/5=0.01.
Interesting. Correction for multiple comparisons is always a good idea with repeated measures contrasts.
I'm a bit confused when you mention p600 and LAN. Are you comparing these in the same anova, and if so why? But I won't delve into this without knowing fully how your comparisons are set up.
if you already know where your effect is localised to (e.g you have a strict region that shows the P600) , and above you say parietal kind of sites, then remember the physiology of EEG here. These electrodes will all likely pick up the same activity, so you could feasibly take the regional average. You'll boost your power massively by reducing the number of levels in the anova. Additionally, if you're hypothesis is to see effects on the left then maybe you could exclude the right hemisphere for now?
Or, as above you could be more exploratory and use fieldtip or the massunivariate toolbox to perform non-parametric clusterbased permutations. However, beware that this might pick up on elements of the signal that are more difficult to explain (not necessarily the peaks).
Alternatively, you could reduce things down even further. Try running ICA on the datasets and then clustering the components. This might reduce down the dataset to a level that could be compared in a much simpler model (e.g 1 component, 2 conditions).
I might have gotten a bit carried away here (or missed the point), but I agree with all of above posts - keep it simple. My input would be don't go chasing significance, and be conservative with your tests. Also, remember to compute appropriate effect sizes and confidence intervals, as these will tell you much more about your effects/interactions than a p value will - regardless of how many corrections, and levels, you have in your model.
If you're really stuck then try running all possible models and see what changes between them. You might surprise yourself.
- Badges
- Science topic
- Similar topics
- Social Psychology
- Attitude