In various experimental paradigms, including the go/no-go paradigm, subjects are not required to make a response after each trial. Instead, they are instructed to respond only when a trial fulfills a certain specification, e.g., contains a signal. I've seen numerous papers report sensitivity indices (d') for such designs, in which a lack of response on a noise trial is treated as a correct rejection. Obviously this conflates true correct rejections with lapses: if the subject was asleep and never responded, say, they would clock up a great correct-rejection rate. Does anyone know of any theoretical studies that have tried to deal with this kind of conflation?
I don't know of any studies. Seems as though the distribution and time history of response speeds (1/rt) on both correct and incorrect detections might provide an index of attentiveness across the test period. Perhaps structure the task's response requirement somewhat like that of the PVT, then categorize response times longer than 500 msec as latent lapses.
I rather doubt that such studies exist, because the experimental paradigm in effect excludes the question. The experimental paradigm in effect assumes
1) that the test is representative of real-world go/no-go discrimination tasks, and
2) the empirical quality of subjects includes all and any factors that may affect their performance, including lapses of attention.
The question you raise, however, implies that either the susceptibility to attention lapses during the test differs from that during real-world go/no-go tasks outside of the test environment, in violation of (1); or the susceptibility to attention lapses is one performance-affecting factor in particular that you want to exclude from the set of all performance-affecting factors faced during the test, in violation of (2).
In practice, of course, assumptions (1) and (2) are never perfectly satisfied, and performance test results differ somewhat from what may occur in real-world. But they differ in a way that generally can never be determined except by (1) analysis of a large set of outcomes for subjects under the actual real-world go/no-task, which data set you do not have of course; or (2) by independent assessment of susceptibility to lapses during your particular test.
Note that your question is not that different from that of pathological subjects who merely guess randomly, or of obstinate subjects who refuse to participate and never actively register a go. The second is usually easy to detect and discard. The first is not so easy.
In any case you can easily assess the expected score for these and other pathological subjects, in order to set benchmarks for various kinds of worst-case/pathological performance. For instance, if the average subject performance is nearly equal to that of the ideal random guesser, then you have probably set a very difficult go/no-go task for subjects. Alternatively, the subjects have misunderstood the instructions given of the test, or are uninterested in it, and so forth.
Or you might compute the correct no-go score assuming a sleeping subject, which gives another benchmark worst-case score for your test against which to compare subjects’ results. This will not tell you if subjects were sleeping at the switch or not, but if they were sleeping, then they would get that score.
These worst-case benchmark performances are always important reference points for assessment of the quality of subjects.
I think that the setting of worst-case performance benchmarks is about as far as theory can take you in the given experimental paradigm. You would have to augment the experiment with countermeasures against attention lapses during the test (breaks, wake-up calls, etc) or with independent experiment of lapse rates in order to remove the impact of lapses from the performance results.
Ronald
Thanks for the responses!
It's disappointing that this doesn't exist. I could imagine there would be some way to estimate, e.g., via Bayesian inference, the true (unmeasured) correct rejection rate based on the probability of lapses, but perhaps more information is needed.
DEAR SAMUEL,
I WOULD ALSO LIKE TO KNOW SPECIFICALLY WHAT YOU INTEND TO MEASURE. I AGREE THAT THE OTHERS HAVE RESPONDED WITH VERY USEFUL, THOUGHTFUL REPLIES. MY EXPERIENCE WITH GO-NO GO TASKS COMES FROM THE PERSPECTIVE OF CLINICAL APPLICATION. I USE GO-NO GO TASKS AS A POSSIBLE INDEX OF "FRONTAL SYSTEM" INTEGRITY/"FRONTAL SYSTEM" INHIBITORY CONTROL. CORRECT RESPONSES, COMMISSION ERRORS, AND RESPONSE LATENCY/REACTION TIMES ARE INTERPRETED WITHIN A PATTERN IN ORDER TO PROVIDE CLINICALLY USEFUL DATA IN INDIVIDUAL CASES. ALL OF THESE VARIABLES FOLLOW A DEVELOPMENTAL TRAJECTORY. THE FREQUENCY OF "TARGET" PRESENTATIONS OCCURS ONLY 13% OF THE TIME; THE HIGHER THE FREQUENCY OF TARGET PRESENTATIONS, THE MORE DIFFICULT IT BECOMES TO ASSESS MY VARIABLES OF INTEREST; WITHIN CLINICAL SETTINGS, TARGET PRESENTATIONS THAT OCCUR OVER 17% OF THE TIME OR THEREABOUTS MAKE IT INCREASINGLY DIFFICULT TO ASSESS MANUAL DISINHIBITION, TYPICALLY AN IMPORTANT CLINICAL FACTOR; HOWEVER, AS OF CIRCA 2000, THERE WERE IN THE NEIGHBORHOOD OF 238 GO-NO GO PARADIGMS, MOST OF THEM EMBEDDED WITHIN CONTINUOUS PERFORMANCE TASKS. FOR THESE REASONS, KNOWING WHAT YOU'D LIKE TO SPECIFICALLY MEASURE WOULD HELP HONE-IN ON UNDERSTANDING YOUR VARIABLES OF INTEREST. AS I CLINICIAN, I'M PRIMARILY INTERESTED IN ASSESSING THE INTEGRITY OF RIGHT HEMISPHERE, VENTRAL FRONTAL-STRIATAL-PALLIDAL-THALAMO-CORTICO "LOOPS," AND PERHAPS MORE SPECIFICALLY, IMPLICATION OF THE "INDIRECT PATHWAY" AS SUGGESTED BY NEUROIMAGING DATA, ALTHOUGH FOR SPECIFIC TYPES OF ERRORS/DISINHIBITION RESPONSES, THERE IS SOME SUGGESTION THAT THE ADJACENT CEREBRO-CEREBELLAR MODULATORY "LOOP" MIGHT REPRESENT A POSSIBLE UNDERPINNING. IF I UNDERSTAND YOUR QUESTION, THE "OMISSION ERRORS" YOU REFER TO, OCCURRING IN HIGH FREQUENCY, ARE PRESUMABLY A MANIFISTATION OF POSTERIOR, PARIETAL INTEGRITY WITHIN THE FRONTO-PARIETAL NETWORK (FPN). ON THE OTHER HAND, IF I'M MISREADING YOUR QUESTION, WHAT I HAVE OFFERRED HERE IS NOTHING MORE THAN CLINICAL GIBBERISH! MICHAEL STEVENS, IN YOUR NECK OF THE WOODS, AT THE INSTITUTE OF LIVING, AND MICHAEL FRANK, AT BROWN, HAVE BOTH WORKED EXTENSIVELY WITH GO-NO GO PARADIGMS, AND THEIR VARIATIONS, AND THEY MIGHT BE ABLE TO ADDRESS YOUR QUESTION MORE DIRECTLY. PLEASE EXCUSE THE "CAPS," - THIS IS FOR MY PURPOSE, NOT INTENDING ANY EMPHASIS WHATSOEVER. - LEN KOZIOL
Thanks for your response Len, however I'm not interested in brain networks or anything like that. I simply want to know if there is some theoretical work on estimating or disentangling trials without responses in go/no-go paradigms. Based on the previous answers it seems the answer is no.
NO PROBLEM; THAT'S EXACTLY WHY I ASKED WHAT I ASKED; IN WHICH CASE, THE REPLY WAS USELESS WITH THE POSSIBLE EXCEPTION OF BEING USEFUL TO A NAIVE CLINICIAN! THANKS. I DO BELIEVE YOU ARE CORRECT - THE ANSWER IS NO. - LEN