Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification

Knoeferle P, Urbach TP, Kutas M (2014)
Acta Psychologica 152: 133-148.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
OA
Autor*in
Knoeferle, PiaUniBi; Urbach, Thomas P.; Kutas, Marta
Abstract / Bemerkung
Extant accounts of visually situated language processing do make general predictions about visual context effects on incremental sentence comprehension; these, however, are not sufficiently detailed to accommodate potentially different visual context effects (such as a scene-sentence mismatch based on actions versus thematic role relations, e.g., (Altmann & Kamide, 2007; Knoeferle & Crocker, 2007; Taylor & Zwaan, 2008; Zwaan & Radvansky, 1998)). To provide additional data for theory testing and development, we collected event-related brain potentials (ERPs) as participants read a subject-verb-object sentence (500 ms SOA in Experiment 1 and 300 ms SOA in Experiment 2), and post-sentence verification times indicating whether or not the verb and/or the thematic role relations matched a preceding picture (depicting two participants engaged in an action). Though incrementally processed, these two types of mismatch yielded different ERP effects. Role-relation mismatch effects emerged at the subject noun as anterior negativities to the mismatching noun, preceding action mismatch effects manifest as centro-parietal N400s greater to the mismatching verb, regardless of SOAs. These two types of mismatch manipulations also yielded different effects post-verbally, correlated differently with a participant's mean accuracy, verbal working memory and visual-spatial scores, and differed in their interactions with SOA. Taken together these results clearly implicate more than a single mismatch mechanism for extant accounts of picture-sentence processing to accommodate. (C) 2014 Elsevier B.V. All rights reserved.
Stichworte
Situated language processing accounts; Sentence-picture verification; Visual context effects; Event-related brain potentials
Erscheinungsjahr
2014
Zeitschriftentitel
Acta Psychologica
Band
152
Seite(n)
133-148
ISSN
0001-6918
eISSN
1873-6297
Page URI
https://pub.uni-bielefeld.de/record/2689395

Zitieren

Knoeferle P, Urbach TP, Kutas M. Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica. 2014;152:133-148.
Knoeferle, P., Urbach, T. P., & Kutas, M. (2014). Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica, 152, 133-148. doi:10.1016/j.actpsy.2014.08.004
Knoeferle, Pia, Urbach, Thomas P., and Kutas, Marta. 2014. “Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification”. Acta Psychologica 152: 133-148.
Knoeferle, P., Urbach, T. P., and Kutas, M. (2014). Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica 152, 133-148.
Knoeferle, P., Urbach, T.P., & Kutas, M., 2014. Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica, 152, p 133-148.
P. Knoeferle, T.P. Urbach, and M. Kutas, “Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification”, Acta Psychologica, vol. 152, 2014, pp. 133-148.
Knoeferle, P., Urbach, T.P., Kutas, M.: Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica. 152, 133-148 (2014).
Knoeferle, Pia, Urbach, Thomas P., and Kutas, Marta. “Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification”. Acta Psychologica 152 (2014): 133-148.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Copyright Statement:
Dieses Objekt ist durch das Urheberrecht und/oder verwandte Schutzrechte geschützt. [...]
Volltext(e)
Access Level
OA Open Access
Zuletzt Hochgeladen
2019-09-06T09:18:25Z
MD5 Prüfsumme
5a5a5e40bbee69bfef1b73999d46fca7


54 References

Daten bereitgestellt von Europe PubMed Central.

The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing
Altmann, Journal of Memory and Language 502–518(), 2007
Incrementality and prediction in human sentence processing
Altmann, Cognitive Science 583–609(), 2009
On the temporal dynamics of language-mediated vision and vision-mediated language
Anderson, Acta Psychologica 181–189(), 2011

Baayen, 2008
Perceptual and symbol systems
Barsalou, Behavioural and Brain Sciences 577–609(), 1999

Bates, 0
The cognitive basis for linguistic structures
Bever, 1970

Carlson, 1989
Sentence comprehension: A psycholinguistic processing model of verification
Carpenter, Psychological Review 45–73(), 1975
On the process of comparing sentences against pictures
Clark, Cognitive Psychology 472–517(), 1972
Working memory span tasks: A methodological review and user's guide
Conway, Psychonomic Bulletin & Review 769–786(), 2005
Situated sentence comprehension: The coordinated interplay account and a neurobehavioral model
Crocker, Brain and Language 189–201(), 2010
An event-related brain potential study of receptive speech comprehension using a modified token test
D'Arcy, Neuropsychologia 1477–1489(), 1999

AUTHOR UNKNOWN, 0
Individual differences in working memory capacity: More evidence for a general capacity theory
Daneman, Memory 6(), 1980
Finding structure in time
Elman, Cognitive Science 179–211(), 1990
Development and preliminary standardization of the “Extended Complex Figure Test” (ECFT)
Fasteneau, Journal of Clinical and Experimental Neuropsychology 18(), 1999

Fasteneau, 2003
Thinking ahead: The role and roots of prediction in language comprehension
Federmeier, Psychophysiology 491–505(), 2007
Good-enough representations in language comprehension
Ferreira, Current Directions in Psychological Science 11(), 2002
The search for “common sense”: An electrophysiological study of the comprehension of words and pictures in reading
Ganis, Journal of Cognitive Neuroscience 89–106(), 1996
Indexical understanding of instructions
Glenberg, Discourse Processes 1–26(), 1999
Grammatical transformations and speed of understanding
Gough, Journal of Verbal Learning and Verbal Behavior 5(), 1965
Interplay between syntax and semantics during sentence comprehension: ERP effects of combining syntactic and semantic violations
Hagoort, Journal of Cognitive Neuroscience 883–899(), 2003
The information conveyed by words in sentences.
Hale J., J Psycholinguist Res 32(2), 2003
PMID: 12690827
The time course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements
Kamide, Journal of Memory and Language 133–156(), 2003
Constructing meaning: The role of affordances and grammatical constructions in sentence comprehension
Kaschak, Journal of Memory and Language 508–529(), 2000
Incremental effects of mismatch during picture–sentence integration: Evidence from eye-tracking
Knoeferle, 2005
The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking
Knoeferle, Cognitive Science 481–529(), 2006
The influence of recent scene events on spoken comprehension: Evidence from eye-movements
Knoeferle, Journal of Memory and Language 519–543(), 2007
Comprehending how visual context influences incremental sentence comprehension: Insights from ERPs and picture–sentence verification
Knoeferle, Psychophysiology 495–506(), 2011
Structure and limited capacity in verbal working memory: A study with event-related potentials
Kolk, Brain and Language 1–36(), 2003
In the company of other words: Electrophysiological evidence for single-word and sentence context effects
Kutas, Language and Cognitive Processes 533–572(), 1993
Thirty years and counting: Finding meaning in the N400 component of the event related brain potential (ERP)
Kutas, Annual Review of Psychology 621–647(), 2011
Event-related brain potentials to semantically inappropriate and surprisingly large words
Kutas, Biological Psychology 99–116(), 1980
Psycholinguistics electrified ii: 1994–2005
Kutas, 2006
Expectation-based syntactic comprehension.
Levy R., Cognition 106(3), 2007
PMID: 17662975
The lexical nature of syntactic ambiguity resolution
MacDonald, Psychological Review 676–703(), 1994
What makes a discourse constraining? Comparing the effects of discourse message and scenario fit on the discourse-dependent N400 effect
Otten, Brain Research 166–177(), 2007
Do people use language production to make predictions during comprehension?
Pickering, Trends in Cognitive Sciences 105–110(), 2007
Examples of mixed-effects modeling with crossed random effects and with binomial data
Quene, Journal of Memory and Language 413–425(), 2008
Achieving incremental semantic interpretation through contextual representation.
Sedivy JC, Tanenhaus MK, Chambers CG, Carlson GN., Cognition 71(2), 1999
PMID: 10444906
Verification of text ideas during reading
Singer, Journal of Memory and Language 574–591(), 2006
The discovery of processing stages: Extensions of Donders' method
Sternberg, Acta Psychologica 276–315(), 1969
NIMH: caught in the line of fire without a general.
Marshall E., Science 268(5211), 1995
PMID: 7732368
Motor resonance and linguistic focus.
Taylor LJ, Zwaan RA., Q J Exp Psychol (Hove) 61(6), 2008
PMID: 18470820
Semantic integration in sentences and discourse: Evidence from the N400
Van, Journal of Cognitive Neuroscience 657–671(), 1999
Monitoring in language perception
Van, Language and Linguistics Compass 1211–1224(), 2009
Monitoring in language perception: Evidence from ERPs in a picture–sentence matching task
Vissers, Neuropsychologia (), 2008
Thematic role assignment in patients with Broca's aphasia: Sentence–picture matching electrified
Wassenaar, Neuropsychologia 716–740(), 2007
Situations models in language comprehension and memory
Zwaan, Psychological Bulletin 162–185(), 1998
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

PMID: 25216075
PubMed | Europe PMC

Suchen in

Google Scholar