Dear Chen,

I am not completely sure whether I am representative in saying this, but... "conducting an experiment against a method" does not seem to convey the meaning/message/question you intended...

Could you please rephrase the question?

(It actually looks like I should be able to give an answer to it, but I am not).

Do you mean, you "employ these methods" in a visual search paradigm? And now you want to know, which of both - reaction time or accuracy - is better to address further questions about visual attention (measured with eye tracking)? (The broad answer for this is, it depends on what your research question is, it always does :))

Best, René

Ps: "Clarify the impact of eye movements by 'the consequence of accuracy'" is also a very confusing sentence, and does not even seem to refer to a real purpose :) (no offense)

René Schlegelmilch

Dear René Schlegelmilch

I have rephrased my question. I hope it is easier to understand my question. You roughly get my idea. From Wolf’s research, it showed that the paradigm of the accuracy of the visual search task could avoid the effect of eye movement. Based on the idea, I would like to conduct one more experiment to clarify my previous experiment to measure RT as the dependent measure. But it doesn't seem a usual way. Thus, I hope to get some suggestions.

Best regards,

Chen

both are critical. Santhi & Reeves (Vis. res, 2004, pages 1235-1256) showed that with parallel processing, (d')^2 / T = cS/ n.Ne + Ni) (equation7)

where

T = processing time = choice RT - simple RT (or 'RT0' ) = RT - RT0

c = stimulus contrast

d' = behavioral sensitivity

n = number of distractors or distractor groups

Ne= noise per distractor

Ni = internal noise, uncorrelated with Ne.

we called (d')^2 / T by the name 'Perf', because it combines the two performance (response) measures. The right-hand side has only stimulus variables.

Note that d' is squared, so changes in accuracy when accuracy is already high have large consequences, often more weighty than changes in RT.

Also note that perform is inverse with T, which assumes that RT does not saturate. Marisa Carrasco showed this is not true for very long RTs; her exponential model, in which RT eventually saturates, does better.

adam reeves

Dear Chen,

now I see. Thank you. From a purely methodological way of addressing questions via comparing two different situations (conditions), your approach seems absolutely valid, where I would only raise two concerns (see below). From a theoretical perspective, I guess, I can not say how plausible your hypothesis is, or 'why' eye movements should affect task performance (because you would need to provide reasons one can evaluate :)). But yes, one can test this hypothesis by 'controlling' eye movements and see whether effects remain or disappear.

Coming to the method, I would see two issues:

1) It seems that this '200ms' stimulus condition, needs a comparison to an 'unconstrained' condition. So maybe it is worth thinking about implementing both conditions again, and then test for an interaction effect, rather than just testing the effect in this 200ms condition only, and then comparing it to a different experiment. This is particularly an advantage if you find 'no interaction' (or no change in the effect, when controlling for eye movements), because, a Null-Effect can only be interpreted in a Bayesian framework (as evidence for or against the Null), for which you need a prior on the interaction effect.

2) I can imagine that there are stronger ways of 'controlling' eye movements. Or - presenting a stimulus only for 200ms does not mean, that participants do not move their eyes anymore. Alternatively, one could use fixation crosses, at which participants have to look at, such that the eyes stand still (which might imply that they would have to do the search task using 'peripheral view', which might not be desirable, on the other hand).

Hope this helps. (Adam Reeves answer looks like there is much more to think about than I am aware of :))

Best, René

Adam Reeves

Dear Dr Reeves

Thanks for your reply. I read the article which was impressive. Your explanation is helpful to understand the article. However, I am still confused about some issues. You stated that changes in accuracy when accuracy is already high have large consequences, often weightier than changes in RT. In traditional RT-oriented experiments, the duration of stimuli is displayed until participants respond. The average accuracy is above 90%. Is it meaningful to compare the accuracy between participants? Have I made a mistake in the design of experiment or analysis? I really hope to apply the signal detection model in my thesis. Thanks in advance.

Regards,

Chen

when accuracy is high, small changes in accuracy can have large effects on d',

as this (in a forced choice situation) is proportional to the z-score of accuracy.

Averaging across subjects is a problem when some of them reach 100% correct, because then d' = infinity. If this happens rarely, standard practice is to replace 100% by 1.0 - 1/(2N), where N is the number of trials. Example; suppose N=100, then 100% is replaced by 99.5%, for a z-score of 2.57. (This method of handling extreme scores was developed in the 1920s and shown to be optimal by Boch and Jones in the 1950s.) If it happens frequently, however, it is better to re-run the experiment using a more difficult condition, so as to avoid the ceiling ( i.e., 100% correct. ) Two people, both scoring 100%, may have quite different levels of ability - the test is just too easy to tell them apart. (Of course, this problem applies to accuracy as well as to d'; high accuracy basically implies, limited information about what is going on. A simple RT should be at 100%, but choice RTs should be less than this.)

This is my opinion. many others disagree; they think that RTs at 100% correct do provide useful information. I believe that such people have not thought carefully about the speed-accuracy trade-off.

adam reeves

René Schlegelmilch

Dear René

Thanks for your reply and your detailed comments. Sorry to reply so late. Regarding the first comments, it is a good point. I guess you state that one is a 200ms condition, and another one is presented until participants respond. I need to find relevant literature to support the idea. Thanks for your suggestions. Although I never involved the null effect, I am reading an article which is concerned with the null effect in the Bayesian framework. The citation is Harms, C., & Lakens, D. (2018). Making “null effects” informative: statistical techniques and inferential frameworks. I hope to add the analysis in my future research. Regarding the second comment, I agree with your opinion about using fixation crosses to control the eye movement. It can gather more direct evidence to investigate whether the result is involved in eye movement.

Regard,

Chen

be careful with fixation crosses. They are a good idea, but they do require attention, because otherwise the eye tends to wander a bit, making small saccades. So one can fixate a cross, but this not only stills the eyes, it also demands attention, as proven in studies of the 'gap effect' (e.g., Jin & Reeves, 2009, in Vision Research). The attentional demands of fixation may, or may not, interact with your other visual test. One can provide a fixation cross and then turn it off 200ms before presenting a 200ms target; in that case, the fixation has gone away, but it is unlikely (though possible) that a small saccade will happen between fixation offset and target offset 400ms later.

Adam Reeves

Dear Dr Reeves,

Thank you for your reply. Sorry for the late response, but I was busy these weeks. Thanks for your detailed explanation and your example concerning accuracy in visual search. Your tips are easy to understand and are practical in the analysis of data from experiments. I will apply your way to handle my data. I agree with your opinion. The speed-accuracy trade-off needs to be considered in visual search tasks. The effect of speed-accuracy trade-off is obvious and intuitive, a scientific way to prove it is not even needed.

Regards,

Chen

Adam Reeves

Dear Dr Reeves,

Thanks for your tips on fixation crosses and gap effect. Recently, I was just thinking about how to use fixation crosses and deciding on gaps of duration between each presentation of stimuli. I am wondering whether there are review articles regarding the fixation cross or gap of duration before the trials began. The tendency of wandering is an interesting issue about which I never thought. I will refer to the article which you mentioned. In addition, I will use the latter design which you mention, whereby avoiding the experiment is affected by small saccades.

Regards,

Chen