Dear all,
Here's one more bid at the great RG force.
I am using Pecher et al.'s (2003) Conceptual Modality Switch paradigm in a psycholinguistic experiment, and I've run into a question for which I would really appreciate your help.
For clarification, a modality switch is a change in perceptual meaning from one stimulus to the next; for instance, from the words 'sour lemon' to 'red flag,' that is, from taste to vision. This change, compared to a transition within the same modality, has been shown to incur processing costs (longer responses) as well brain signals of harder comprehension (N400 event-related potential).
Traditionally, the second word presented (whether noun or adjective) has been the point of measure, both for RTs and ERPs. Yet, could it be better to measure at the first word?
Theoretically, it should be beneficial because any modality shift from one trial to the next should arise at the first rather than the second word of the second trial, as there is greater novelty there. Now, the criticism I have already garnered against a first-word measure argues that any perceptual content will only be strong enough after two consistent instances of a given modality, and so the measure has to be taken afterwards. How do you see it?
There are several perks to measuring at the first word. First, the effect can be tracked from its start through all stages in the time course of word processing. Second, it would cancel confound influence. The participants' task in modality-switch experiments is usually to judge the relationship between a property and a concept, whether true (sour / lemon), or false (sweet / lemon). In this sense, measuring at the first word would cancel confound influence on the N400 from the mere relationship between each noun and adjective. In contrast, the second word takes that in as participants start to ponder the relationship between the two words. Third, measuring at the first word solves the issue of response artifacts in the EEG, as responses are only taken after the second word. Therefore it allows better for a fast-response task, as no artifact-shield (a 1-second pause) is required after the second word.
Thank you in advance for any ideas, based on literature or your experience. Best regards,
Pablo
.........................................
Most relevant references:
Collins, J., Pecher, D., Zeelenberg, R., & Coulson, S. (2011). Modality Switching in a Property Verification Task: An ERP Study of What Happens When Candles Flicker after High Heels Click. Frontiers in Psychology, 2.
Hald, L. A., Marshall, J.-A., Janssen, D. P., & Garnham, A. (2011). Switching Modalities in A Sentence Verification Task: ERP Evidence for Embodied Language Processing. Frontiers in Psychology, 2.
Lynott, D., & Connell, L. (2009). Modality exclusivity norms for 423 object properties. Behavior Research Methods, 41, 2, 558-564.
Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying different-modality properties for concepts produces switching costs. Psychological Science, 14, 2, 119-24.
Solomon, K. O., & Barsalou, L. W. (2004). Perceptual simulation in property verification. Memory & Cognition, 32, 244-259.
Van Dantzig, S., Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2008). Perceptual Processing Affects Conceptual Processing. Cognitive Science, 32, 579-590.
Conference Paper Modality switch effects emerge early and increase throughout...
Pablo, you have made an good argument for measuring on the first word. It would be interesting to see what the difference is when the modality switch impact is measured on the first word rather than, as is typical, on the second word. Have you considered running a pilot in which you compare the results with subjects randomly assigned to either a first word measurement or a second word measurement condition? The direction of the differences (if you find differences) might be informative about the processing factors you mention.
Hi Judith,
Thank you so much! That comparison had crossed my mind too, yet it's not feasible for me now, as this is my master's project. I shall keep it in mind.
Best regards
First or second word in the measure of modality switch? It all depends on the syntactic structure of the language involved. In your examples Pablo, you consider 'Sour Lemon', 'Red Flag' from English which has the adjective in the 'pre-head' position; in this case the choice of first word will focus on the quality of the referent, and not the referent itself. It goes without saying that entities and objects are basic representations within our mind, while qualities and attributes are indeed secondary aspects which are part of these representations. In your examples LEMON and FLAG are these basic entities, with SOUR and RED as attributes of secondary salience. A credible measure of Modality switch must, therefore, take cognizance of the structure of the source language, the marked-unmarked dynamics regarding the possibility, or otherwise, of yoking given words together within common usage in a given language.
Hi Yakobo,
The testing language is critical indeed. In my case, the experiment is in Dutch, yet the linguistic research informing it is from English. Most relevant are modality norms for nouns and adjectives (by Lynott and Connell), where adjectives were found to be more exclusive in their principal modality than nouns. It is for that reason that I'm presenting the adjective before the noun in each trial. Dutch and English are most likely analogous in this respect.
This connects to the issue of cross-linguistic generalizability in psycholinguistics: a rarely-discussed elephant in the room. Interesting to see, so thank you!
Best regards
Hello Judith, Yakobo, and everyone else,
As an update on this question from long ago, I would like to share the results of this experiment, which finally did measure the conceptual modality switch at the first word of the target trial. The results revealed a switching effect that started at the P2 component and extended thereafter with an increasing tendency. The design, the results in detail, and the entire data set are available through OSF (https://osf.io/97unm/). Also, a paper will be published this summer in the Conference of the Cognitive Science Society (https://osf.io/preprints/psyarxiv/a5pcz).
Thanks a lot
Conference Paper Modality switch effects emerge early and increase throughout...
- Badges
- Science topic
- Similar topics
- Psychology
- Cognitive Science