View in EDS

(Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension

Saved in:
Bibliographic Details
Title: (Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension
Language: English
Authors: Cohn, Neil, Paczynski, Martin, Jackendoff, Ray
Source: Cognitive Psychology. Aug 2012 65(1):1-38.
Availability: Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Peer Reviewed: Y
Physical Description: PDF
Page Count: 38
Publication Date: 2012
Document Type: Journal Articles
Reports - Research
Descriptors: Sentences, Semantics, Syntax, Language Processing, Semiotics, Experiments, Visual Stimuli, Comparative Analysis, Interaction, Sentence Structure, Comprehension, Reading Comprehension
DOI: 10.1016/j.cogpsych.2012.01.003
ISSN: 0010-0285
Abstract: Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning. (Contains 9 figures and 4 tables.)
Abstractor: As Provided
Entry Date: 2012
Accession Number: EJ963642
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ963642
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: (Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Cohn%2C+Neil%22">Cohn, Neil</searchLink><br /><searchLink fieldCode="AR" term="%22Paczynski%2C+Martin%22">Paczynski, Martin</searchLink><br /><searchLink fieldCode="AR" term="%22Jackendoff%2C+Ray%22">Jackendoff, Ray</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Cognitive+Psychology%22"><i>Cognitive Psychology</i></searchLink>. Aug 2012 65(1):1-38.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: PhysDesc
  Label: Physical Description
  Group: PhysDesc
  Data: PDF
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 38
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2012
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Sentences%22">Sentences</searchLink><br /><searchLink fieldCode="DE" term="%22Semantics%22">Semantics</searchLink><br /><searchLink fieldCode="DE" term="%22Syntax%22">Syntax</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Processing%22">Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Semiotics%22">Semiotics</searchLink><br /><searchLink fieldCode="DE" term="%22Experiments%22">Experiments</searchLink><br /><searchLink fieldCode="DE" term="%22Visual+Stimuli%22">Visual Stimuli</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Interaction%22">Interaction</searchLink><br /><searchLink fieldCode="DE" term="%22Sentence+Structure%22">Sentence Structure</searchLink><br /><searchLink fieldCode="DE" term="%22Comprehension%22">Comprehension</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Comprehension%22">Reading Comprehension</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.cogpsych.2012.01.003
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0010-0285
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning. (Contains 9 figures and 4 tables.)
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2012
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ963642
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ963642
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.cogpsych.2012.01.003
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 38
        StartPage: 1
    Subjects:
      – SubjectFull: Sentences
        Type: general
      – SubjectFull: Semantics
        Type: general
      – SubjectFull: Syntax
        Type: general
      – SubjectFull: Language Processing
        Type: general
      – SubjectFull: Semiotics
        Type: general
      – SubjectFull: Experiments
        Type: general
      – SubjectFull: Visual Stimuli
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Interaction
        Type: general
      – SubjectFull: Sentence Structure
        Type: general
      – SubjectFull: Comprehension
        Type: general
      – SubjectFull: Reading Comprehension
        Type: general
    Titles:
      – TitleFull: (Pea)nuts and Bolts of Visual Narrative: Structure and Meaning in Sequential Image Comprehension
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Cohn, Neil
      – PersonEntity:
          Name:
            NameFull: Paczynski, Martin
      – PersonEntity:
          Name:
            NameFull: Jackendoff, Ray
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2012
          Identifiers:
            – Type: issn-print
              Value: 0010-0285
          Numbering:
            – Type: volume
              Value: 65
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Cognitive Psychology
              Type: main
ResultId 1