Based on the ERP results, statistically identifying differences between groups/conditions will require us to run parametric/non-parametric tests. So if those tests show there is a significant difference then perhaps there is. Maybe you can look into the data set again and see whether it need some additional data processing or something that might have missed during processing. Then re-run the appropriate statistical test again.

Hope this helps. Cheers!

Potentially of interest if you have high-density EEG: Article Nonparametric statistical testing of EEG- and MEG-data

Hi Hiran Perera-W.A.

Thank you so much for your help!

Ineed, in some phenomenons the finding we can get theoretically and mathematically, might not practically interpretable in a real environment. Therefore, it is critical to select the most suitable way to justify and explain the results accordingly. This is the key point in my opinion.

I appreciate your time to answer my question.

Cheers,

Reza

Hi Tom A Campbell

Thank you so much for your time and answer!

Both materials you have shared were useful, I am going to make some tests in order to make clear the results. Though, sometimes it is difficult to discuss with reviewers what you want to express without providing proper tests on results.

Kind regards,

Reza

Dear Reza,

It is not fully clear what data you compare. Is that ERPs to different kinds of stimuli or with different kinds of filtering?? You do not mention the sampling rate either. In conventional EEG recordings, you have a sampling rate of 1000 Hz (or less). For such recordings, you would consider an error of 2 ms (=2 sampling points) as of little importance, given the much larger interindividual variance you usuallly have in such data. Nevertheless, you mention that you got "SD= 2ms, max difference =4ms min_difference=0ms". This means that all (!) your differences were >0 ms. Given this, any non-parametric test will indicate a significant difference. Whether such a difference is also meaningful is a different question.

All the best, Timm

Hi Timm Rosburg

Thank you so much for your time and reply!

I am sorry for unclarity, Now I have revised and added some information about the test I would like to do.

In ERP error detection, the resolution of data could play an important role as you mentioned. The point is, I do not prefer to do a test for finding significant p-value in which I will need to change alpha to make the results I expect. instead of that, maybe the equivalence test can be more applicable in this case.

Cheers,

Reza

For an equivalence test, you need to define equivalence bounds. You have probably already checked the literature on this topic (e.g. Walker & Nowacki, 2011, J. Gen. Intern. Med., doi: 10.1007/s11606-010-1513-8).

However, equivalence of ERP data might refer to different qualities (amplitude, time, frequency, phase). Thus, it might be more practical to subtract the estimated signal from the actually recorded signal in the time and frequency domain and to visualize the output. If these difference potentials/spectra look essentially like random noise and the noise level is below the noise level of the original recordings, you will presumably confirm this in an equivalence test. If the difference potentials / spectra do have systematic features, your signals are likely not equivalent.

Thanks for your time and replying Timm Rosburg

I have tried your suggestion and it seems the results support our hypothesis now, however, I got some difficulties about determining the "d" parameter (i.e. [-d d]).

I have defined it [-5 5] ms as the accuracy of the test for my case study(I will be thankful if you could help me to determine this parameter).

Since the data is simulated data, I have defined a silent noise (20dB) for testing the sensitivity of my method.

What do the marked ranges meaning (i.e. in equivalence interval)? does anyone know how to explain those ranges?