What you are describing is what the clinicians call Visual Field testing (specifically: Static Perimetry). Though it sounds like you are only using a single stimulus level, rather than trying to estimate a detection threshold (which may or may not be a good idea).

In the clinical field, standard practice is to use Yes/No, for the practical reasons you describe. You can also randomise the timing of the presentation window, and/or include explicit False Positive / False Negative catch trials, to detect any unusual behaviour.

Depending on your stimuli/hypothesis, you may also be able to able to do something 'clever', like ask people to report the orientation of the stimulus, rather than its presence (thereby allowing for a 1-interval, m-alternative forced-choice). However, got to be careful that this is appropriate for your purposes, e.g., see Thibos (1998): http://www.opt.uh.edu/onlinecoursematerials/PHOP6241/Bedell/Fall%202014/Thibos%20Acuity%20Perimetry%20OVS_98.pdf

The criterion effects in Yes/No are real, but are generally not a barrier to getting good, clean data. Certainly it is considered 'good enough' by the clinicians, who run millions (billions?) of such tests a year. Whether it is good enough for you will depend on exactly what you're trying to show, and may also be more/less important depending on your population/protocol though (are these undergrads who will be tested for many hours, or 4 year old children who will be tested for minutes?)

Finally, may be worth noting that no paradigm, including mAFC, is really criterion/bias free: (see Jones 2015: http://psycnet.apa.org/record/2015-15457-001 ), so the idea that 'yes/no' bad, 'mAFC' good is open to question anyway. Ultimately, whether something is good enough for you is largely an empirical question.

Hi, Pete R Jones

Thank you very much for the detailed answer, but I still have some remaining questions:

(1). Can you please explain more on "explicit False Positive / False Negative catch trials" ? Like how to include them in experiment in practice, and why can they detect unusual behavior?

(2). Do you know whether here are any papers that systematically study the effect of stimulus presentation length on visual field testing? How will the sensitivity or threshold result differ if I briefly present the stimulus for 100/200/300 ms?

Thanks.

False positive = impossible stimuli that they should never answer "yes" to.

False negative = very easy (suprathreshold) stimuli that they should never answer "no" to.

If some individuals have strangely high values for either, could be a warning sign

"How will the sensitivity or threshold result differ if I briefly present the stimulus for 100/200/300 ms?" -- what kindof stimulus are you using?

Hi, Pete R Jones ,

The stimulus is a small (0.3 degree) sinusoidal grating embedded in a noise background (15 degree). Subject need to judge whether the grating is present or not. I have not measure the detection threshold yet, but if the grating can be easily seen if it is placed at fixation point.

I don't know if your use of noise changes anything, but in quiet anything above 100 msec shouldn't make much difference: Article Temporal studies with flashed gratings: Inferences about hum...

In clinical perimetry, durations tend to be either 100 msec or 200 msec, with little difference in results between the two (e.g., Article The effect of stimulus duration upon the components of fluct...

It seems to me that it is hardly necessary to add something to the Pete Jones' comprehensive replies, but I will try to further clarify some small points.

You said you would like to measure d', but actually none of the three paradigms you mentioned is suitable for that, since they are are threshold measurement methods. My guess is you are aware of this fact and seek to avoid the effort that the application of signal detection theory requires.

Paradigm III cannot be used, though. The reason is not only that it fails at the fixation point, but that it contradicts your the premises of your experiment. You want to measure sensitivity in different positions of the visual field, which means you assume that sensitivity is a function of location. On the other hand a 2AFC task is based on the assumption of shift invariance, i.e it does not matter where the stimulus is presented because sensitivity is the same at both chosen places. Otherwise the guessing probability would be confounded with the location dependent sensitivity.

As it has already been hinted at, if you were to pursue the forced choice approach you could make the two alternatives qualitatively different so that you have two orientations or two different colors or whatever in the same location. However, this requires to make sure that the qualitative diferences do not influence sensitivity, which might be the case for grating orientation.

Personally I would stick with paradigm II and ignore the erroneous responses about which your subjects reported. I assume they are equally probable for both time intervals, so instead of 50% guessing probality you have a pedastal 50%+x

below your psychometric curve, and the "threshold" would be shifted to 75%+x/2. However, because in the light of signal detection theory these 75% are just a more or less reasonable convention, we may as well leave it with that.

As to the stimulus presentation times only very short ones would lead to higher thresholds due to Bloch's law. You may want to check that out.

Hi Detlef Ruschin ,

Thanks for your reply, but I am a bit confused about the second paragraph in your answer. I know that these methods are threshold measurement methods, but in my experiment I plan to fix the noise background contrast and the sine grating contrast, and test how good subject can detect the sine grating at 25 retinal locations (by 2IFC, as you suggest). At each location the 2IFC task will be repeated for several times, so I can get a 25 d' at 25 locations. These d' will be interpolated by a surface function, and form a d' map across the visual field, and the d' map will be used later in another experiment, which use the same stimulus (same target and noise contrast) as here.

What is the problem with this experiment? My understanding is that if I have a couple of Hit and False Alarm trials I can calculate d', and I don't think a psychometric curve is needed here. Do I have some misunderstanding about signal detection theory?

I may not have fully understood your experimental design, but to measure d' you would not only have to repeat the task several times, but you would have to manipulate to subjects' decision criterion and to generate a sensitivity curve by connecting the hits over false alarms values for each criterion. Then d' is the distance of that curve from the diagonal. In case you are just calcuting the 75% quantile of the correct answers in the repeated task, you are applying the legacy threshold theory but not signal detection theory.

A note on the background noise. You are adding that if you are interested in the issue of masking. If masking is not your reseach topic than you would want the grating to be a modulation of a plain gray patch.

Hi Detlef Ruschin ,

I hope to do a quick replication of this old Nature article: https://www.nature.com/articles/nature03390

But I find that the visibility map of their original experiment is quite long, and I hope if there is some way to simplify it.

The authors measure 4 psychometric curves (for 4 noise levels) at each of the 25 locations, and I am wandering whether it is indeed needed.

Thanks for pointing to this article. I see you are estimating d' from the percentage of correct resonses as mentioned in Green & Swets p.108. I cannot further comment on the possibility to reduce the number of signal/noise combinations, but modelling search as a matching procedure makes only sense for known target features, doesn't it? It might be that in the normal case saliency models provide a better explanation for the selection of fixation locations. My two pennies.

"old nature article" -- 2005! =)

Looks good. But the way to speed it up if you don't want to fit whole psychometric

functions is to use an adaptive procedures (ideally QUEST, which you can find implemented in PsychToolbox [MATLAB] and PsychoPy [Python]). That said, if your participants are able to bear it, I'd just do what Najemnik & Geisler did, as it's much less likely to go wrong (and will be much quicker to implement/debug).

Some additional points of clarification:

Detlef Ruschin -- a full ROC is unnecessary if one is willing/able to assume that all noise is Gaussian and I.I.D. (probably not true, but not the kind of thing a reviewer is likely to complain about).

Detlef Ruschin -- Mh Stein wouldn't/shouldn't be using % correct per se, but rather the proportion of correct responses on each interval, which should be much the same as % correct, but would account for any interval bias.

Mh Stein -- Fixing stimulus intensity and measuring d' at a single point (rather than a threshold) is unlikely to work for the simple practical reason because you'll likely run into floor/ceiling effects (I.e., the task may be too hard or too easy for some locations/observers).

As to floor/ceiling, using a combination of target and noise parameters that has a high position (but below 100%) on one the curves in fig.1b from the nature article it might be possible that the respective signal is still sometimes detected when it is presented in the periphery rather than in the fovea. However, I find this difficult to predict because of the way in which the data are presented. A quick preliminary trial might be necessary to clarify this.

Just to provide the original formulation for a quick d' estimate, which Mh Stein seems to have taken note of during his review of the literature:

Green & Swets is obviously great, but I'd suggest that Ch. 6 of Wickens (Elementary SDT) is slightly better and easier to follow in this instance (particularly with regards to avoiding interval bias). There he gives the standard equation, but also a slightly improved formulation.

Picking a high but non-100% position might be possible. However, the published curves are presumably for a small number of highly-trained individuals. There is likely a non-trivial degree of individual variability, particularly in more naive observers, in which case thresholds (or whole psychometric functions) will be necessary.