View in EDS HTML Full Text PDF Full Text

A Starring Role for Inference in the Neurocognition of Visual Narratives

Saved in:
Bibliographic Details
Title: A Starring Role for Inference in the Neurocognition of Visual Narratives
Language: English
Authors: Cohn, Neil (ORCID 0000-0003-0074-1995)
Source: Cognitive Research: Principles and Implications. 2021 6.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 14
Publication Date: 2021
Document Type: Journal Articles
Reports - Research
Descriptors: Inferences, Cognitive Processes, Brain, Visual Learning, Cartoons, Schemata (Cognition), Brain Hemisphere Functions, Language Processing
DOI: 10.1186/s41235-021-00270-9
ISSN: 2365-7464
Abstract: Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped "action stars" where a reader "knows" events are undepicted "at that moment," rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a "noise" panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300-500 ms), but action stars and noise panels then differed in their later effects (500-900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.
Abstractor: As Provided
Entry Date: 2021
Accession Number: EJ1287610
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFGz8ZTBcKj_3dvY7i4jmhQAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDPP7oYI4JHb2MEkddAIBEICBmyd9eDZ_cxjlvSLE3sK889BrjchM-wZuKHk1z9nG7DBk2qe69G74Av4-vwtKMpo2H0Po5wljCJT-NzGEu-7bVKo9Bqb3U9cfeRtMLd-VDsgqVez9bm9fMmBos8ykkbDk0rEsOhzT5FcGImgG9AxlzyqfZA--3R9d01QeGjGlGvKq3F1WvJ2LKRKdkivnPdU9LBhix5uE_FtqBkcI
Text:
  Availability: 1
  Value: <anid>AN0148720207;[k1e6]15feb.21;2021Feb17.02:43;v2.2.500</anid> <title id="AN0148720207-1">A starring role for inference in the neurocognition of visual narratives </title> <p>Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped "action stars" where a reader knows events are undepicted at that moment, rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a "noise" panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300–500 ms), but action stars and noise panels then differed in their later effects (500–900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.</p> <p>Keywords: Visual language; Inference; Visual narrative; Discourse; Comics</p> <hd id="AN0148720207-2">Introduction</hd> <p>Inference has been a primary focus of studies of discourse (McNamara and Magliano [<reflink idref="bib50" id="ref1">50</reflink>]), especially in research on how we understand visual narratives, like comics or picture stories (Cohn [<reflink idref="bib11" id="ref2">11</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref3">41</reflink>]; McCloud [<reflink idref="bib48" id="ref4">48</reflink>]; Saraceni [<reflink idref="bib61" id="ref5">61</reflink>]). Research on verbal narrative has often used visual narratives as stimuli, assuming them to involve or evoke similar inferential processing (Gernsbacher et al. [<reflink idref="bib23" id="ref6">23</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref7">41</reflink>]; Magliano et al. [<reflink idref="bib43" id="ref8">43</reflink>]), and visual narratives have even been posited as tools for bootstrapping verbal inferential abilities (Kendeou et al. [<reflink idref="bib32" id="ref9">32</reflink>]). Yet, little research exists on the neurocognition of visual narrative inferencing itself. Thus, studying how inferences are generated in sequential images can be informative for understanding such processing across domains. This study thus asks: what are the neurocognitive correlates of inference generation, particularly when an impoverished narrative unit explicitly signals omitted information?</p> <p>Much work on inferential processing in both modalities has emphasized bridging inferences, where a reader must infer the absent event information from what is explicitly provided. This is usually taken as a <emph>backward-looking process</emph>: a reader realizes the discontinuity between the incoming information and the prior context, and then works to "fill in" that missing information (McNamara and Magliano [<reflink idref="bib50" id="ref10">50</reflink>]). Such backward-looking processes have long been demonstrated in research on verbal discourse (McNamara and Magliano [<reflink idref="bib50" id="ref11">50</reflink>]), and comparable inferential processing has been posited across both verbal and visual narratives (Gernsbacher et al. [<reflink idref="bib23" id="ref12">23</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref13">41</reflink>]; Magliano et al. [<reflink idref="bib43" id="ref14">43</reflink>]). Indeed, as in studies of verbal discourse (McKoon and Ratcliff [<reflink idref="bib49" id="ref15">49</reflink>]), costs related to inferential processing manifest at the image following omitted-information as longer viewing times and increased visual search processes (Huff et al. [<reflink idref="bib30" id="ref16">30</reflink>]; Hutson et al. [<reflink idref="bib31" id="ref17">31</reflink>]; Magliano et al. [<reflink idref="bib45" id="ref18">45</reflink>], [<reflink idref="bib44" id="ref19">44</reflink>]). Such costs are modulated by interference to both linguistic and visuospatial working memory processes (Magliano et al. [<reflink idref="bib45" id="ref20">45</reflink>]), again suggesting a connection between the mechanisms involved in verbal and visual narratives.</p> <p>However, consider Fig. 1b, where the penultimate panel uses the visual narrative convention of an "action star" (Cohn [<reflink idref="bib8" id="ref21">8</reflink>], [<reflink idref="bib10" id="ref22">10</reflink>]), whereby the climactic event (Lucy hitting Charlie with a beat-up baseball) remains unseen (as in Fig. 1a). With an action star, a reader <emph>knows</emph> an absent climactic event occurs <emph>at that moment,</emph> rather than events being omitted with no climatic unit at all. Action stars thus differ from situations using the canonical backward-looking bridging inferences because no image is "missing," but rather a visual morpheme implies an event without showing it. This star shape typically depicts an impact or collision, but also can signal loudness when surrounding text like an onomatopoeia, which often co-occurs with action stars (Manfredi et al. [<reflink idref="bib47" id="ref23">47</reflink>]). As they are conventionalized, action stars are a panel-level unit "lexicalized" within the visual language used in comics that allow authors to leave out information and thereby sponsor reader engagement via the resulting inference (and/or providing a way for artists to avoid drawing complex climactic events). Such engagement has been posited to enhance readers' immersion and likeability of narratives (Herman [<reflink idref="bib27" id="ref24">27</reflink>]; McCloud [<reflink idref="bib48" id="ref25">48</reflink>]; Zwaan [<reflink idref="bib71" id="ref26">71</reflink>]).</p> <p>Graph: Fig. 1 Visual narrative with either an a) explicit climactic event, or its substitution by b) an action star which omits the event but still provides a narrative climax. Peanuts is © Peanuts Worldwide LLC</p> <p>Yet, action stars do not just signal missing content, they also play a specific role in a sequence, since they omit climactic events specifically. <emph>Visual Narrative Grammar</emph> (VNG) posits that a narrative structure operates in parallel to semantics, where it organizes meaning at a discourse level (Cohn [<reflink idref="bib8" id="ref27">8</reflink>], [<reflink idref="bib11" id="ref28">11</reflink>]). The canonical narrative schema progresses through narrative roles where the climactic moment—the Peak—contains the crucial information of the sequence. This importance is suggested because participants choose to omit Peaks at lower rates, and recognize Peaks when missing at higher rates, than other narrative categories (Cohn [<reflink idref="bib9" id="ref29">9</reflink>]; Magliano et al. [<reflink idref="bib44" id="ref30">44</reflink>]).</p> <p>Action stars thus fulfil a narrative role as a Peak, letting the sequencing structure remain well-formed with a unit devoted to a climax, while still pushing the reader to generate an inference for the un-depicted events. This structural role is suggested by viewing times that are actually <emph>faster</emph> for action stars than blank panels in Peak position when both omit events (Cohn and Wittenberg [<reflink idref="bib18" id="ref31">18</reflink>]). Since action stars play a narrative role as Peaks, they facilitate processing more readily than "incongruous" panels with no content at all. Yet, consistent with other studies of visual narrative inference, panels following action stars are viewed slower than those following explicit events, implying a greater cost for construing the omitted events. Thus, both action stars and event-omission may warrant a backward-looking bridging inference at the subsequent panel, but action stars provide an explicit cue that signals to a reader that inference generation is necessary, but without omitting a narrative unit.</p> <p>Nevertheless, the neurocognition of such inferencing remains understudied, though research on event-related brain potentials (ERPs) implicate similar mechanisms in visual narratives as in language processing (Cohn [<reflink idref="bib11" id="ref32">11</reflink>]). When encountering a panel in a visual narrative sequence, a comprehender first extracts the relevant information from a panel (Loschky et al. [<reflink idref="bib40" id="ref33">40</reflink>]). Specific visual cues are identified as relevant for the understanding of the sequence, given the prior context (Foulsham et al. [<reflink idref="bib22" id="ref34">22</reflink>]; Hutson et al. [<reflink idref="bib31" id="ref35">31</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref36">41</reflink>]). These cues might be part of characters, especially faces and postures (Foulsham and Cohn [<reflink idref="bib21" id="ref37">21</reflink>]; Hutson et al. [<reflink idref="bib31" id="ref38">31</reflink>]; Laubrock et al. [<reflink idref="bib37" id="ref39">37</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref40">41</reflink>]), but in action stars they are the lines comprising this abstract symbol.</p> <p>Information extracted from the visual surface then activates semantic memory to process the meaning of the immediate representation in relation to what came before in the sequence. This process is indexed in ERPs by the N400, a negative deflection that peaks around 400 ms after the onset of a word or image (Kutas and Federmeier [<reflink idref="bib35" id="ref41">35</reflink>]; Kutas and Hillyard [<reflink idref="bib36" id="ref42">36</reflink>]). The N400 has been taken to reflect the brain's default process of semantic access and/or integration, modulated by the expectancy of an incoming stimulus given the preceding context (Baggio [<reflink idref="bib1" id="ref43">1</reflink>]; Hagoort [<reflink idref="bib24" id="ref44">24</reflink>]; Kuperberg [<reflink idref="bib33" id="ref45">33</reflink>]; Kutas and Federmeier [<reflink idref="bib35" id="ref46">35</reflink>]; Nieuwland et al. [<reflink idref="bib54" id="ref47">54</reflink>]). As a result, larger N400 effects are shown to incongruous or unexpected images in a visual narrative sequence (or words in a sentence) than congruous ones (Coderre et al. [<reflink idref="bib7" id="ref48">7</reflink>]; West and Holcomb [<reflink idref="bib69" id="ref49">69</reflink>]). Indeed, greater N400s have been shown to bridging inferences in studies of language (Kuperberg et al. [<reflink idref="bib34" id="ref50">34</reflink>]; St. George et al. [<reflink idref="bib63" id="ref51">63</reflink>]). Though the N400 is taken to index a modality-general mechanism, visual images typically evoke another negativity peaking around 300 ms, preceding the N400, deemed the N300 (McPherson and Holcomb [<reflink idref="bib51" id="ref52">51</reflink>]). The N300 has been associated with object categorization or identification (Hamm et al. [<reflink idref="bib26" id="ref53">26</reflink>]) prior to the general semantic access indexed by the N400 (Draschkow et al. [<reflink idref="bib20" id="ref54">20</reflink>]; Hamm et al. [<reflink idref="bib26" id="ref55">26</reflink>]; Võ and Wolfe [<reflink idref="bib68" id="ref56">68</reflink>]).</p> <p>The information activated in semantic memory then becomes incorporated into a <emph>situation model</emph> (McNamara and Magliano [<reflink idref="bib50" id="ref57">50</reflink>]; van Dijk and Kintsch [<reflink idref="bib66" id="ref58">66</reflink>]), a growing mental understanding of the unfurling scene. With each unit of a (visual) discourse, comprehenders may anticipate subsequent information, both for broad expectancies (ex. expecting the same characters to appear in subsequent frames) and for more specific ones (ex. predicting specific subsequent events). As the narrative progresses, discontinuities trigger an update to integrate or revise the situation model with this new information. Greater changes thus lead to increased updating across shifts between characters, spatial locations, events, and other aspects of the discourse (Zwaan and Radvansky [<reflink idref="bib72" id="ref59">72</reflink>]).</p> <p>While situation model construction progresses in an ongoing and incremental manner across the units of (visual) narratives (Huff et al. [<reflink idref="bib29" id="ref60">29</reflink>]; Magliano and Zacks [<reflink idref="bib46" id="ref61">46</reflink>]), more significant discontinuities may warrant further processing. These cases include contexts where a reader may be motivated to fill in information that is not provided directly, thereby leading to the generation of inference. For example, in Fig. 1b, upon reaching the final panel, an inference would be expected to resolve the incongruity of the missing climactic event. Thus, bridging inferences can be viewed as a process of situation model construction triggered in the absence of information provided overtly.</p> <p>In ERPs, updating or revision processes have been indexed by the P600, a positive deflection posteriorly distributed across the scalp that typically onsets after the time window of the N400 (Brouwer et al.; Kuperberg [<reflink idref="bib33" id="ref62">33</reflink>]; Van Petten and Luka [<reflink idref="bib67" id="ref63">67</reflink>]). Though originally evoked by syntactic violations or ambiguities in language (Hagoort et al. [<reflink idref="bib25" id="ref64">25</reflink>]; Osterhout and Holcomb [<reflink idref="bib55" id="ref65">55</reflink>]), P600s have now been associated with processes related to the revision or integration of a structure given the relation between incoming information and its previous context (Baggio [<reflink idref="bib1" id="ref66">1</reflink>]; Brouwer et al. [<reflink idref="bib6" id="ref67">6</reflink>]; Kuperberg [<reflink idref="bib33" id="ref68">33</reflink>]), possibly tied to more general cognitive updating processes (Donchin and Coles [<reflink idref="bib19" id="ref69">19</reflink>]; Leckey and Federmeier [<reflink idref="bib38" id="ref70">38</reflink>]). In visual narratives, P600s have been observed to both incongruous and congruous changes between panels to events, characters, or the framing of information (Cohn [<reflink idref="bib11" id="ref71">11</reflink>]; Cohn and Foulsham [<reflink idref="bib12" id="ref72">12</reflink>]).</p> <p>Nevertheless, P600s may not index inference processes specifically. Only one ERP study has investigated inference processing in visual narratives (Cohn and Kutas [<reflink idref="bib14" id="ref73">14</reflink>]), which compared sequences showing explicit events with those showing a character watching an event off-panel (such as Charlie Brown watching Snoopy chase after a ball he threw). A final panel then confounded the expectations of the narrative sequence (like Linus returning with the ball in his mouth instead of Snoopy). Here, the "onlooker" (Cohn [<reflink idref="bib10" id="ref74">10</reflink>]) of the off-panel event either cued a climactic Peak (e.g., with a surprised expression) or lacked such cues (e.g., with a neutral facial expression). The cued-Peaks evoked a larger P600 than those without such a cue, taken to reflect greater updating of a situation model given the explicitness of the cued-observer's actions, and indeed even larger P600s appeared to explicit events. Thus, while the P600 may reflect an updating of the situation model, it cannot solely indicate inferential processes at this position, as it appears even greater to non-inference demanding representations, all of which were congruous in the sequence context.</p> <p>Indeed, at the subsequent panel which clarified the off-panel event, a P600 was suggested only following the non-cued onlooker, despite both cued and non-cued versions eliciting the same inference. The P600 in this context was taken as an indicator of the difference in narrative structure, since the inferential content was held constant, with the preceding cued panel being more suggestive of a climactic Peak than the non-cued panel. Indeed, P600s have appeared to disruptions of the narrative grammar alone, with no manipulations of meaning (Cohn et al. [<reflink idref="bib13" id="ref75">13</reflink>]) suggesting a process of structural reanalysis or revision in line with P600s originally shown to violated syntactic structure sentence processing (Hagoort et al. [<reflink idref="bib25" id="ref76">25</reflink>]; Osterhout and Holcomb [<reflink idref="bib55" id="ref77">55</reflink>]). Such findings further support that the P600 may operate across both grammatical and semantic processing, or at their interface (Bornkessel-Schlesewsky and Schlesewsky [<reflink idref="bib3" id="ref78">3</reflink>]; Brouwer et al. [<reflink idref="bib6" id="ref79">6</reflink>]; Michalon and Baggio [<reflink idref="bib52" id="ref80">52</reflink>]).</p> <p>Rather, inferential processes in this study may have instead been indexed by sustained anterior negativities which were evoked by panels following both cued and non-cued onlookers (Cohn and Kutas [<reflink idref="bib14" id="ref81">14</reflink>]). Such sustained negativities often with a more central or frontal distribution, have been posited in studies of verbal discourse to reflect interpretive processes subsequent to the relational semantics indexed by the N400 (Baggio [<reflink idref="bib1" id="ref82">1</reflink>]; Baggio et al. [<reflink idref="bib2" id="ref83">2</reflink>]; Bott [<reflink idref="bib5" id="ref84">5</reflink>]; Hoeks and Brouwer [<reflink idref="bib28" id="ref85">28</reflink>]). Such negativities may index working memory processes seeking to resolve complex ambiguities, like inferences (van Berkum [<reflink idref="bib65" id="ref86">65</reflink>]), or a mechanism searching through a mental model (Hoeks and Brouwer [<reflink idref="bib28" id="ref87">28</reflink>]). Sustained negativities have been observed to visual narratives following the N400 time window in most all studies of semantic processing looking at sequential images (Cohn et al. [<reflink idref="bib17" id="ref88">17</reflink>]; West and Holcomb [<reflink idref="bib69" id="ref89">69</reflink>]).</p> <p>In comparison with onlookers, action stars show no explicit referential information at all—they depict only a symbolic star-shape—while directly cuing both narrative and inference (Cohn [<reflink idref="bib10" id="ref90">10</reflink>]). Thus, here we compared sequences depicting explicit events with those substituted with action stars. These conventional sequences were further contrasted with sequences substituting climactic events with "noise panels"—panels created by scrambling the lines from action stars into non-representational configurations (Fig. 2).</p> <p>Graph: Fig. 2 Example stimuli for visual narrative sequences with a critical panel of either an explicit event, a conventionalized action star, or a noise panel. Peanuts is © Peanuts Worldwide LLC</p> <p>Given prior theories of visual narrative processing (Cohn [<reflink idref="bib11" id="ref91">11</reflink>]; Loschky et al. [<reflink idref="bib41" id="ref92">41</reflink>]), the action star should demand relatively few costs of information extraction, because of its simple symbolic representation, compared to more complex depictions of events. Such information extraction would thus be impossible for noise panels, which depict no meaningful cues at all. From here, semantic access (N400) should become harder for action stars and noise panels, because the surface information will have less feature overlap with explicitly depicted events. If the event-cues of action stars sponsor an inference more than the noise panels, and if such an inference is manifest in the access to semantic memory, we might expect an attenuated N400 to action stars. However, both the action star and noise panels could equally sponsor inference at this position, as in findings that N400s did not differ to onomatopoetic (<emph>Pow!</emph>) and descriptive words (<emph>Punch!</emph>) in action stars, which were both attenuated compared to anomalous words, suggesting that they were both sufficient as event-cues given the context of the prior visual sequence (Manfredi et al. [<reflink idref="bib47" id="ref93">47</reflink>]).</p> <p>Regardless of their relational semantics, the event-cues of an action star should better contribute to constructing a situation model than a noise panel, which cues no semantic information at all. Thus, a greater P600 should appear to action stars than the noise panel, indexing the cost of updating or revising a situation model. Such a finding would be consistent with prior work showing a larger P600 to onlookers with cues suggestive of off-panel events compared to those without such cues (Cohn and Kutas [<reflink idref="bib14" id="ref94">14</reflink>]). This may provide evidence that action stars trigger an update of information into a situation model, even if content is not provided outright.</p> <p>We might also predict updating processes at the panel following an action star or noise panel, as further explicit information should demand the inference of the missing content. This has been suggested behaviorally by longer viewing times following action stars and the omission of climactic events (Cohn and Wittenberg [<reflink idref="bib18" id="ref95">18</reflink>]; Hutson et al. [<reflink idref="bib31" id="ref96">31</reflink>]; Magliano et al. [<reflink idref="bib45" id="ref97">45</reflink>]). Nevertheless, prior work has not shown a P600 to inference-following panels in visual narratives, where a P600 appeared only to a <emph>narrative</emph> difference between panels where the inference was held constant. Rather, a frontally distributed negativity appeared to both types of inference generating panels, taken as possibly indicative of working memory processes, in line with sustained negativities in language to sentence- and discourse-level inferences (Baggio [<reflink idref="bib1" id="ref98">1</reflink>]; Baggio et al. [<reflink idref="bib2" id="ref99">2</reflink>]; Bott [<reflink idref="bib5" id="ref100">5</reflink>]; Hoeks and Brouwer [<reflink idref="bib28" id="ref101">28</reflink>]). Thus, sustained negativities at the panel following action stars or noise panels would also be consistent with inferential processing.</p> <hd id="AN0148720207-3">Materials and methods</hd> <p></p> <hd id="AN0148720207-4">Stimuli</hd> <p>We used visual sequences constructed out of panels from <emph>The Complete Peanuts</emph> which have been used in prior studies of visual narrative comprehension (e.g., Cohn et al. [<reflink idref="bib17" id="ref102">17</reflink>]; Cohn and Wittenberg [<reflink idref="bib18" id="ref103">18</reflink>]). All sequences were 6-panels long, had no text, and were normed to a consistent size. To ensure that these sequences had strong potential for inference of a missing Peak panel, our stimulus selection was guided from data from a norming study. There, 101 participants (54 female; mean age: 40.8, range: 23–65; mean VLFI: 20.8, range 2.5–49) viewed 120 sequences where one panel had been omitted, and they were asked to identify where a panel had been missing. This allowed us to assess the rate by which participants may be accurately inferring missing content, relevant here for Peak panels specifically. We therefore chose 60 sequences from this prior study to use in this experiment. Our final sequences had a mean of 0.71 (SD = 0.16) accuracy for participants recognizing that the Peak panel was omitted.</p> <p>From these strips, we created three sequence types, as in Fig. 2. Our event-<emph>explicit</emph> sequences used the original sequence with Peak panels intact. Critical panels fell in various ordinal positions so that participants could not predict its location, including the second panel in the sequence (2 strips), third panel (<reflink idref="bib5" id="ref104">5</reflink>), fourth panel (<reflink idref="bib13" id="ref105">13</reflink>), and fifth panel (<reflink idref="bib40" id="ref106">40</reflink>). Action star sequences substituted an action star panel for the explicit Peak panels. Because action star panels can have less visual information than explicit panels, we therefore designed action star panels with a similar degree of visual complexity as our explicit Peak panels. We calculated the "gray value" of all pixels across all explicit Peak panels (<emph>M</emph> = 213.2) and designed three action star panels with similar values (<emph>M</emph> = 213.0). These three action star panels were distributed randomly across sequences within the action star sequence types. Our final sequence type then created "noise" panels by selecting subdivisions of action star panels (via a grid and iterated circular pixel selections), and then rotating these subdivisions randomly such that no discernable representational information was conveyed, though the quantity and character of the visual lines remained the same. Action stars were used as the base for noise panels rather than explicit panels in order to ensure they remained asemantic, and also involved similar line segments. Noise panels had a grey value comparable to other critical panels (<emph>M</emph> = 213.0).</p> <p>These 60 sequences were counterbalanced into three lists using a Latin Square Design such that participants viewed each sequence only once, but across lists all participants viewed all sequences in all conditions. An additional 96 fillers, half of which contained sequencing discontinuities, were used to introduce additional heterogeneity into the experimental stimuli.</p> <hd id="AN0148720207-5">Participants</hd> <p>We recruited 24 participants from Tilburg University (14 male, 10 female; mean age: 23.8, SD = 4.9). Although statistical power was not computed a priori, a power analysis in G*Power indicated that with 3 conditions in a sample of 24 participants, to achieve a medium effect size of 0.25, it would require <emph>F</emph>-values above 3.2 for our within-subjects design. All participants gave their informed written consent prior to the experiment and the study was approved by the Tilburg University School of Humanities and Digital Sciences Research Ethics and Data Management Committee. Participants were right-handed, taking no psychiatric medications, with no history of head trauma, and with normal or corrected-to-normal vision. Prior to experimentation, participants filled out the Visual Language Fluency Index (VLFI) questionnaire which assessed their frequency of reading and drawing visual narratives like comics, along with experiencing other media like movies and written books. The VLFI score generated by this assessment has been a consistent predictor of individual differences in visual narrative comprehension (Cohn and Kutas [<reflink idref="bib14" id="ref107">14</reflink>]; Cohn and Maher [<reflink idref="bib16" id="ref108">16</reflink>]; Cohn et al. [<reflink idref="bib17" id="ref109">17</reflink>]; Cohn and Wittenberg [<reflink idref="bib18" id="ref110">18</reflink>]). Participants in this study overall had a mean VLFI score of 14.14 (SD = 5.7, range 3–24.7), which is considered average (high: > 20, average: ~ 12, low: < 8) and consistent with average scores in prior ERP studies.</p> <hd id="AN0148720207-6">Procedure</hd> <p>The experiment was conducted in a soundproof chamber, where a participant sat in a chair across from a computer screen. Lights in the chamber were turned off, except for backlighting behind the screen used to prevent a flashing effect of stimuli on the screen that could induce blinks. We used PsychoPy2 (Peirce et al. [<reflink idref="bib57" id="ref111">57</reflink>]) to present experimental trials, which began with a screen reading "Ready" in white letters on a grey background, where participants pressed a button to start each trial. A red dot persisted in the center of the screen to provide participants with a sustained fixation point. Once participants pressed to begin a trial, each black-and-white panel appeared one at a time in the center of the otherwise grey screen at a size of 10.16 × 8.04 cm, yielding a visual angle of 5.2° horizontally and 4.2° vertically. Each panel remained on screen for a duration of 1350 ms, separated by a 300 ms ISI to prevent a "flip book" effect of panels appearing to be animated. These durations are consistent with prior ERP studies of visual narratives (Cohn et al. [<reflink idref="bib13" id="ref112">13</reflink>]; Cohn and Kutas [<reflink idref="bib14" id="ref113">14</reflink>], [<reflink idref="bib15" id="ref114">15</reflink>]; Cohn and Maher [<reflink idref="bib16" id="ref115">16</reflink>]). At the end of each sequence, a question mark cued participants to rate the comprehensibility of the sequence on a 1 (= hard to understand) to 7 (= easy to understand) Likert scale.</p> <p>Following all experimental trials, participants filled out a post-test questionnaire which probed their observations of stimuli, which they could fill in with open-response answers. This questionnaire asked: "Did you notice any patterns in the comic strips that you saw? Did you notice anything unusual about any of the comic strips? If yes, what do you think made them unusual? Were there aspects of these unusual strips that were different from each other? Why do you think they were different (or similar)?".</p> <hd id="AN0148720207-7">Data analysis</hd> <p>EEG was recorded using a Brain Products ActiChamp system at a sampling rate of 250 Hz and high cutoff filter of 70 Hz. EEG recordings were made with 32 channel Standard actiCAPs, referenced online to electrode Fz. Eye movements and blinks were monitored using electrodes placed beneath the right eye and beside the left eye. Electrode impedances for all electrodes were kept below 10 kΩ. We analyzed the data using the ERPLAB plugin for EEGLAB in MATLAB (Lopez-Calderon and Luck [<reflink idref="bib39" id="ref116">39</reflink>]). EEG data was refiltered offline with a bandpass filter of 0.1–30 Hz, and re-referenced offline to the average of the mastoid channels (TP9, TP10). An additional filter of 0.1–15 Hz was applied for presentation of data in Figs. 3 and 4, but this filtered data was not analyzed.</p> <p>Trials with excessive blink or muscle artifact were isolated and removed. Across conditions at the critical panels, 6% (range 3–8%) of trials were rejected per participant, an average of 1.25 trials (18.75 trials retained). A 3 (explicit, action star, noise panel) × 2 (critical panel, critical panel + 1) ANOVA showed no significant differences in the rejection rates across sequence types (<emph>p</emph> = 0.246), critical panel position (<emph>p</emph> = 0.095), or their interaction (<emph>p</emph> = 0.210).</p> <p>Our analysis focused on the ERPs to the critical, manipulated panel, and to the subsequent, critical panel + 1. ERP amplitudes across sequence types were compared in the epochs of 200–300 ms, 300–500 ms, 500–800 ms, and 800–1100 ms corresponding to the ERP components of the N300, N400, and later effects (sustained negativities, P600, late frontal positivity). Additional analyses in epochs of 0–100 ms and 100–200 ms investigated the potential for early, stimulus-driven physical differences at the critical panels. A broad coverage of the scalp was examined using five regions of interest which each averaged the amplitudes across four electrodes. As in Figs. 3 and 4, these included a central region (FC1, FC2, CP1, CP2), and peripheral regions of the left anterior (Fp1, F7, F3, FC5), right anterior (Fp2, F8, F4, FC6), left posterior (CP5, P3, P7, O1), and right anterior (CP6, P4, P8, O2). Statistical analyses for each critical panel and each epoch used repeated-measures ANOVAs with factors of Sequence Type (3 levels: explicit, action star, noise panel), and additional factors in the peripheral regions of Hemisphere (2 levels: left, right) and Anterior–Posterior (AP) Distribution (2 levels: anterior, posterior). Significant interactions in the omnibus analyses of peripheral regions were followed by repeated measures ANOVAs comparing Sequence Types in each region with post-hoc pairwise contrasts using a Bonferroni correction.</p> <p>Graph: Fig. 3 ERPs time-locked to the critical panel of explicit event panels, action stars, and noise panels represented across electrodes and topographic maps. Highlights show analyzed epochs of the primary effects of the N400, P600, and Late Frontal Positivity. Negative is plotted upwards</p> <p>Graph: Fig. 4 ERPs time-locked to the critical panel + 1, i.e., the panel following explicit event panels, action stars, and noise panels represented across individual electrodes and topographic maps. Highlight shows the analyzed epoch of the sustained negativity. Positivity Negative is plotted upwards</p> <p>Behavioral results were compared using a repeated-measures ANOVA with three levels of Sequence Type (explicit, action star, noise), with post hoc tests using a Bonferroni correction. Finally, to investigate any possible influence of comic reading experience, we used Pearson's correlations with an alpha level set to 0.05 between VLFI scores and ratings and the mean amplitude differences averaged across all electrode sites at the critical and subsequent panels.</p> <hd id="AN0148720207-8">Results</hd> <p></p> <hd id="AN0148720207-9">Behavioral results</hd> <p>Analysis of participants' ratings suggested differences in the sequences' comprehensibility, <emph>F</emph>(<reflink idref="bib2" id="ref117">2</reflink>,<reflink idref="bib46" id="ref118">46</reflink>) = 12.54, <emph>p</emph> < 0.001. Explicit sequences (<emph>M</emph> = 5.3, SD = 1.1) were rated as more comprehensible than sequences with action stars (<emph>M</emph> = 4.5, SD = 0.94) or noise panels (<emph>M</emph> = 4.6, SD = 0.87), all <emph>t</emph>s > 4, all <emph>p</emph>s < 0.005, but these latter types did not differ from each other (<emph>p</emph> = 1). Nevertheless, all sequences were rated as comprehensible on the whole, rated above the midway point of 4 on a 1–7 scale (all <emph>t</emph>s > 2.6, all <emph>p</emph>s < 0.05). Finally, in open response post-test questionnaires, 35% of participants commented on observing action stars, while 33% noticed noise panels and that they differed from action stars (ex. "some pictures were more like an explosion, others looked like wires"), including one who drew a noise panel.</p> <hd id="AN0148720207-10">Critical panel</hd> <p>We first analyzed ERPs at the critical panels, depicted in Fig. 3. All results from omnibus analyses are provided in Table 1. In the 0–100 ms epoch, an interaction arose between Sequence Type and Hemisphere. However follow up analyses with ANOVAs in each region resulted in main effects of Sequence Type which did not exceed the threshold of significance (all <emph>F</emph>s < 2.7, all <emph>p</emph>s > 0.079, all <emph>η</emph><sups>2</sups><subs>p</subs> < 0.105). No significant differences arose in the 100–200 ms epoch.</p> <p>Table 1 Results of ANOVAs comparing sequence types at the critical panel and critical panel + 1 for central and peripheral regions of interest</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="3" /><th align="left" colspan="4"><p>Sequence type (ST)</p></th><th align="left" colspan="2"><p>ST*H</p></th><th align="left" colspan="2"><p>ST*AP</p></th><th align="left" colspan="2"><p>ST*H*AP</p></th></tr><tr><th align="left" colspan="2"><p>Central</p></th><th align="left" /><th align="left" /><th align="left" colspan="6"><p>Peripheral</p></th></tr><tr><th align="left"><p><italic>F</italic></p></th><th align="left"><p><italic>η</italic><sup>2</sup><sub><italic>p</italic></sub></p></th><th align="left"><p><italic>F</italic></p></th><th align="left"><p><italic>η</italic><sup>2</sup><sub><italic>p</italic></sub></p></th><th align="left"><p><italic>F</italic></p></th><th align="left"><p><italic>η</italic><sup>2</sup><sub><italic>p</italic></sub></p></th><th align="left"><p><italic>F</italic></p></th><th align="left"><p><italic>η</italic><sup>2</sup><sub><italic>p</italic></sub></p></th><th align="left"><p><italic>F</italic></p></th><th align="left"><p><italic>η</italic><sup>2</sup><sub><italic>p</italic></sub></p></th></tr></thead><tbody><tr><td align="left"><p><italic>Critical panel</italic></p></td><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>0–100</p></td><td char="." align="char"><p>1.9</p></td><td char="." align="char"><p>0.08</p></td><td char="." align="char"><p>1.9</p></td><td char="." align="char"><p>0.08</p></td><td char="." align="char"><p>4.4*</p></td><td char="." align="char"><p>0.16</p></td><td char="." align="char"><p>0.3</p></td><td char="." align="char"><p>0.013</p></td><td char="." align="char"><p>0.7</p></td><td char="." align="char"><p>0.03</p></td></tr><tr><td align="left"><p>100–200</p></td><td char="." align="char"><p>0.48</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p>0.04</p></td><td char="." align="char"><p>0.002</p></td><td char="." align="char"><p>0.03</p></td><td char="." align="char"><p>0.001</p></td><td char="." align="char"><p>0.99</p></td><td char="." align="char"><p>0.04</p></td><td char="." align="char"><p>0.39</p></td><td char="." align="char"><p>0.02</p></td></tr><tr><td align="left"><p>200–300</p></td><td char="." align="char"><p>6.8**</p></td><td char="." align="char"><p>0.23</p></td><td char="." align="char"><p>1.7</p></td><td char="." align="char"><p>0.07</p></td><td char="." align="char"><p>18.3***</p></td><td char="." align="char"><p>0.44</p></td><td char="." align="char"><p>0.54</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p><.001</p></td></tr><tr><td align="left"><p>300–500</p></td><td char="." align="char"><p>7.3**</p></td><td char="." align="char"><p>0.24</p></td><td char="." align="char"><p>7.6***</p></td><td char="." align="char"><p>0.25</p></td><td char="." align="char"><p>8.8***</p></td><td char="." align="char"><p>0.28</p></td><td char="." align="char"><p>0.4</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p>0.15</p></td><td char="." align="char"><p>0.01</p></td></tr><tr><td align="left"><p>500–800</p></td><td char="." align="char"><p>1.1</p></td><td char="." align="char"><p>0.05</p></td><td char="." align="char"><p>1.6</p></td><td char="." align="char"><p>0.06</p></td><td char="." align="char"><p>7.4**</p></td><td char="." align="char"><p>0.24</p></td><td char="." align="char"><p>2.1</p></td><td char="." align="char"><p>0.08</p></td><td char="." align="char"><p>3.5*</p></td><td char="." align="char"><p>0.13</p></td></tr><tr><td align="left"><p>800–1100</p></td><td char="." align="char"><p>1.5</p></td><td char="." align="char"><p>0.06</p></td><td char="." align="char"><p>1.3</p></td><td char="." align="char"><p>0.05</p></td><td char="." align="char"><p>8.1***</p></td><td char="." align="char"><p>0.26</p></td><td char="." align="char"><p>2.4</p></td><td char="." align="char"><p>0.09</p></td><td char="." align="char"><p>0.15</p></td><td char="." align="char"><p>0.01</p></td></tr><tr><td align="left"><p><italic>Critical panel</italic> + <italic>1</italic></p></td><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /><td char="." align="char" /></tr><tr><td align="left"><p>200–300</p></td><td char="." align="char"><p>0.21</p></td><td char="." align="char"><p>0.01</p></td><td char="." align="char"><p>0.26</p></td><td char="." align="char"><p>0.01</p></td><td char="." align="char"><p>0.75</p></td><td char="." align="char"><p>0.03</p></td><td char="." align="char"><p>0.27</p></td><td char="." align="char"><p>0.01</p></td><td char="." align="char"><p>0.33</p></td><td char="." align="char"><p>0.01</p></td></tr><tr><td align="left"><p>300–500</p></td><td char="." align="char"><p>0.43</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p>0.21</p></td><td char="." align="char"><p>0.009</p></td><td char="." align="char"><p>3.6*</p></td><td char="." align="char"><p>0.13</p></td><td char="." align="char"><p>0.91</p></td><td char="." align="char"><p>0.04</p></td><td char="." align="char"><p>0.59</p></td><td char="." align="char"><p>0.03</p></td></tr><tr><td align="left"><p>500–800</p></td><td char="." align="char"><p>0.42</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p>0.89</p></td><td char="." align="char"><p>0.04</p></td><td char="." align="char"><p>6.5**</p></td><td char="." align="char"><p>0.22</p></td><td char="." align="char"><p>4.2*</p></td><td char="." align="char"><p>0.15</p></td><td char="." align="char"><p>0.02</p></td><td char="." align="char"><p><.001</p></td></tr><tr><td align="left"><p>800–1100</p></td><td char="." align="char"><p>1.7</p></td><td char="." align="char"><p>0.07</p></td><td char="." align="char"><p>1.9</p></td><td char="." align="char"><p>0.08</p></td><td char="." align="char"><p>6.6**</p></td><td char="." align="char"><p>0.22</p></td><td char="." align="char"><p>2.7^</p></td><td char="." align="char"><p>0.1</p></td><td char="." align="char"><p>0.77</p></td><td char="." align="char"><p>0.03</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>ST</emph> sequence type, <emph>H</emph> hemisphere, <emph>AP</emph> anterior–posterior distribution ^ <emph>p</emph> <.1; * <emph>p</emph> <.05; ** <emph>p</emph> <.01; *** <emph>p</emph> <.001. <emph>df</emph> = 2,46</p> <p>In the 200–300 ms epoch, a main effect of Sequence Type arose in the central region and interactions in the peripheral regions were found between Sequence Type and Hemisphere. Follow up ANOVAs in each region revealed main effects of Sequence Type in both anterior regions (all <emph>F</emph>s > 7.6, all <emph>p</emph>s < 0.001, all <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> > 0.25) and in the right posterior region, <emph>F</emph>(<reflink idref="bib2" id="ref119">2</reflink>,<reflink idref="bib46" id="ref120">46</reflink>) = 3.3, <emph>p</emph> < 0.05, <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> = 0.13. Pairwise comparisons suggested that noise panels evoked greater negativities than both explicit panels and action stars in the central and anterior regions (all <emph>p</emph>s < 0.05).</p> <p>In the 300-500 ms epoch, main effects of Sequence Type appeared in both central and peripheral regions. An additional Sequence Type × Hemisphere interaction was followed by analyses revealing main effects in both anterior regions (all <emph>p</emph>s > 8.4, all <emph>p</emph>s < 0.001, all <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> > 0.42). Across central and anterior regions, pairwise comparisons showed a greater negativity indicative of an N400 appeared to action stars and noise panels than explicit panels (all <emph>p</emph>s < 0.05), but no difference between action stars and noise panels.</p> <p>Later effects were first suggested in the 500-800 ms epoch by Sequence Type × Hemisphere and Sequence Type × Hemisphere × AP Distribution interactions in the peripheral regions. Main effects of Sequence Type were subsequently found in both anterior regions (all <emph>F</emph>s > 6.7, all <emph>p</emph>s < 0.01, all <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> > 0.149). Pairwise contrasts suggested that action stars evoked a greater negativity than both explicit and noise panels in the left anterior region (all <emph>p</emph>s < 0.05), and a greater negativity than the noise panels alone in the right anterior region (<emph>p</emph> < 0.05).</p> <p>Additional differences were suggested in the 800–1100 ms epoch with an interaction between Sequence Type and Hemisphere. Follow up ANOVAs suggested main effects of Sequence Type in the left anterior and right posterior regions (all <emph>F</emph>s > 3.9, all <emph>p</emph>s < 0.05, all <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> < 0.14). Pairwise comparisons in the left anterior region suggested that noise panels evoked a frontal positivity that was close but did not exceed the threshold of significance compared to action stars (<emph>p</emph> = 0.050) and explicit panels (<emph>p</emph> = 0.058). In the right posterior region, post-hoc comparisons implicated a greater posterior positivity to action stars than noise panels (<emph>p</emph> < 0.05), which is more visible in the topographic maps of Fig. 3 than in the waveforms. No differences were found between action stars and explicit panels.</p> <hd id="AN0148720207-11">Critical panel + 1</hd> <p>At the panel following the manipulated, critical panel, no differences occurred between sequence types in the 200–300 ms epoch, but interactions between Sequence Type and Hemisphere appeared in the 300–500, 500–800, and 800–1100 ms epochs (Table 1), and an interaction between Sequence Type and AP Distribution appeared in the 500–800 ms epoch. Follow up analyses revealed no significant effects in regions in the 300–500 ms epoch, but main effects of Sequence Type appeared in the right posterior region for both the 500–800 and 800–1100 ms epochs (all <emph>F</emph>s > 6.9, all <emph>p</emph>s < 0.005, all <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> < 0.23) and in the left posterior region of the 800–1100 ms epoch, <emph>F</emph>(<reflink idref="bib2" id="ref121">2</reflink>,<reflink idref="bib46" id="ref122">46</reflink>) = 3.6, <emph>p</emph> < 0.05, <emph>η</emph><sups>2</sups><subs><emph>p</emph></subs> = 0.138. These differences arose because action stars and noise panels evoked a larger rightward posterior negativity than explicit panels (all <emph>p</emph>s < 0.05), which extended into the left posterior for only noise panels compared to explicit panels (<emph>p</emph> < 0.05).</p> <hd id="AN0148720207-12">Individual differences</hd> <p>The analysis of expertise correlated only marginally with behavioral results. A positive correlation between ratings of explicit sequences and VLFI scores suggested that greater comic reading frequency lead to higher ratings, <emph>r</emph>(<reflink idref="bib22" id="ref123">22</reflink>) = 0.47, <emph>p</emph> < 0.05, but when correcting for multiple comparisons this correlation was no longer significant. No significant correlations were found for sequences with action stars or noise panels, nor the differences between them. No significant correlations were observed between VLFI scores and ERP results.</p> <hd id="AN0148720207-13">Discussion</hd> <p>Inference has been a hallmark of the study of visual narrative processing. Here, we asked about such processing both for representationally impoverished action stars and noise panels and at the subsequent image. Our first apparent finding was that panels omitting information in a sequence elicit a large N400 effect compared to explicit depictions of an event. These N400s were evoked both by the highly conventionalized action stars and the not-conventionalized noise panels. That these inexplicit panels did not differ in amplitude of the N400 implies that they both trigger a greater demand for accessing semantic memory given the absence of overt representations. Under one interpretation, this N400 could indicate inferential processing, accessing the semantic features of the unseen information (Kuperberg et al. [<reflink idref="bib34" id="ref124">34</reflink>]; St. George et al. [<reflink idref="bib63" id="ref125">63</reflink>]). However, a simpler interpretation is that this response reflects the bottom-up activation of semantic memory (Baggio [<reflink idref="bib1" id="ref126">1</reflink>]; Kuperberg [<reflink idref="bib33" id="ref127">33</reflink>]; Kutas and Federmeier [<reflink idref="bib35" id="ref128">35</reflink>]) relative to the content in the prior panels, potentially in reference to expectancies of subsequent actions (Coderre et al. [<reflink idref="bib7" id="ref129">7</reflink>]; Magliano et al. [<reflink idref="bib42" id="ref130">42</reflink>]). As the bottom-up semantic content of the action stars and noise panels overlap minimally with prior panels, they trigger large N400 effects compared to explicit depictions of events.</p> <p>Note that if such panels were viewed as purely asemantic, we would expect them to generate attenuated N400s compared to depictions of explicit events. This would be comparable to the findings of the lack of an N400 for action stars containing non-word symbols ($#*!) when compared to onomatopoetic or event-descriptive words in lieu of explicit events in visual narratives (Manfredi et al. [<reflink idref="bib47" id="ref131">47</reflink>]). Our observed effects also differ from those to blank frames that disrupted the narrative constituent structure of visual sequences, and in turn elicited left anterior negativities more associated with grammatical processing (Cohn et al. [<reflink idref="bib13" id="ref132">13</reflink>]). The evocation of large N400s here by action stars and noise panels suggests that meaning is being triggered, though it incurs greater cost than explicitly depicted events. However, these responses in the context of this experimental design cannot confirm whether such N400s are indicative of an inference per se.</p> <p>Given that the explicit events, action stars, and noise panels differed in their representations, it was possible the N400 could have been affected by stimulus-driven physical differences between these panels. Though we attempted to control for physical differences by balancing grey values across critical panels, significant differences were found in the earlier 0–100 ms and 200–300 ms epoches. We cannot rule out the possibility that early differences affected later components. However, these earlier patterns of effects (0–100: explicit > noise/action star; 200–300: noise > explicit > action star) appear different than those observed at later epochs. In addition, despite differences prior to 300 ms, action stars and noise panels had similar amplitudes and latencies of the N400 on visual inspection, including topologically in its phasic ascent and descent. If such stimulus differences sponsored downstream effects, then these relative differences do not seem consistent across time windows in a sustained way (cf. Rossion and Caharel [<reflink idref="bib60" id="ref133">60</reflink>]; Taylor et al. [<reflink idref="bib64" id="ref134">64</reflink>]).</p> <p>Despite similar N400 effects, the action star panels and noise panels differed in their subsequent processing. A late posterior positivity appeared for action stars in the later epoch, consistent with the P600 (Brouwer et al. [<reflink idref="bib6" id="ref135">6</reflink>]; Leckey and Federmeier [<reflink idref="bib38" id="ref136">38</reflink>]). Posterior positivities have been implicated in the updating of a situation model in visual narratives, for both congruous and incongruous situational changes between panels and events (Cohn [<reflink idref="bib11" id="ref137">11</reflink>]; Sitnikova et al. [<reflink idref="bib62" id="ref138">62</reflink>]). As these positivities occurred only to action stars—just like the concurrent sustained negativity—it potentially reflects the updating and/or reanalysis of the situation model of the sequence built by the context, given the incoming information of the action star (Baggio [<reflink idref="bib1" id="ref139">1</reflink>]; Brouwer et al. [<reflink idref="bib6" id="ref140">6</reflink>]; Kuperberg [<reflink idref="bib33" id="ref141">33</reflink>]; Van Petten and Luka [<reflink idref="bib67" id="ref142">67</reflink>]). Indeed, larger P600s have been observed to congruous panels with more explicit situational information (Cohn and Kutas [<reflink idref="bib14" id="ref143">14</reflink>]), prior to recognition of the need for an inference. However, if such a positivity reflects such conceptual integration, it would remain fairly unspecific for action stars ("an event happens") and would still warrant further inferencing to resolve.</p> <p>An alternative interpretation is that this positivity evoked only by action stars reflects a process of reanalyzing this panel's role as a Peak in the narrative structure, whether or not integrating into a situation model (Cohn [<reflink idref="bib11" id="ref144">11</reflink>]; Cohn et al. [<reflink idref="bib13" id="ref145">13</reflink>]). This would align with P600s appearing to syntactic structure in language, and proposals that the P600 reflects processing at the interface of syntax and semantics (Brouwer et al. [<reflink idref="bib6" id="ref146">6</reflink>]; Michalon and Baggio [<reflink idref="bib52" id="ref147">52</reflink>]). Such a structural interpretation may be supported in that action stars and noise panels also differed in a left anterior distribution in the 500–800 ms epoch. This distribution is consistent with the left anterior negativity evoked by violations of narrative grammar (Cohn [<reflink idref="bib11" id="ref148">11</reflink>]; Cohn et al. [<reflink idref="bib13" id="ref149">13</reflink>], [<reflink idref="bib17" id="ref150">17</reflink>]), which also have appeared along with P600s (Cohn et al. [<reflink idref="bib13" id="ref151">13</reflink>]; Cohn and Kutas [<reflink idref="bib15" id="ref152">15</reflink>]), and with similar ERPs as evoked by violations of syntactic structure in language (Bornkessel-Schlesewsky and Schlesewsky [<reflink idref="bib4" id="ref153">4</reflink>]; Neville et al. [<reflink idref="bib53" id="ref154">53</reflink>]). Both interpretations (situation model updating, narrative structure) are possibly supported by this posterior positivity only differing from noise panels, but not explicit panels. Differentiating these interpretations would require further experimentation, such as with action stars placed in other positions in a sequence.</p> <p>A "lexicalized" interpretation for action stars is further implicated by their contrast with the late effects evoked by noise panels. Between 800 and 1100 ms, noise panels evoked a late frontal positivity (LFP) that was greater than both action stars and explicit panels. In research on language, LFPs have been associated with violated lexical or semantic predictions, such as when a particular word is expected in the sentence, but participants are presented with a different, yet congruent word (Leckey and Federmeier [<reflink idref="bib38" id="ref155">38</reflink>]; Van Petten and Luka [<reflink idref="bib67" id="ref156">67</reflink>]). In visual narratives, frontal positivities have been elicited in both congruous and incongruous contexts, yet with unexpected or improbable frequency (Cohn and Foulsham [<reflink idref="bib12" id="ref157">12</reflink>]; Cohn and Kutas [<reflink idref="bib15" id="ref158">15</reflink>]; Cohn and Maher [<reflink idref="bib16" id="ref159">16</reflink>]). Substitutions of words into image sequences also yield LFPs. When action stars contain words that index unseen events, descriptive "sound effects" (<emph>Punch!</emph>) elicited larger LFPs than onomatopoetic sound effects (<emph>Pow!</emph>), regardless of congruity (Manfredi et al. [<reflink idref="bib47" id="ref160">47</reflink>]). Such descriptive sound effects as a class appear less frequently than onomatopoeia in comics (Pratha et al. [<reflink idref="bib59" id="ref161">59</reflink>]), making them categorically less probable. That is, within the context of visual narratives, LFPs appear not only to congruous-but-unexpected stimuli, but to stimuli of any type of congruency with low probability of occurrence.</p> <p>This LFP could thus be interpreted as noise panels being unexpected in the context of a visual sequence. By comparison, the lack of an LFP to action stars may affirm their conventionality as substitutive panels within a visual narrative sequence (Cohn [<reflink idref="bib10" id="ref162">10</reflink>]). Such findings are similar to the faster self-paced viewing times shown to action stars than empty panels, despite having more visual complexity (Cohn and Wittenberg [<reflink idref="bib18" id="ref163">18</reflink>]). To the extent that similar LFPs appear across sentences, visual sequences, and multimodal interactions between text and image, it could imply a domain-general response sensitive to probability of incoming forms in a given context (Leckey and Federmeier [<reflink idref="bib38" id="ref164">38</reflink>]; Van Petten and Luka [<reflink idref="bib67" id="ref165">67</reflink>]). This would align with proposals that LFPs are tied to general mechanisms like the P300, which is sensitive to attentional and probabilistic processing (Donchin and Coles [<reflink idref="bib19" id="ref166">19</reflink>]; Leckey and Federmeier [<reflink idref="bib38" id="ref167">38</reflink>]; Polich [<reflink idref="bib58" id="ref168">58</reflink>]). It is thus possible that, consistent with the P3a (Polich [<reflink idref="bib58" id="ref169">58</reflink>]), the LFP here reflects an attentional response to the unlikely, and unfamiliar, noise panels, which contrast from the conventionally recognized action stars.</p> <p>It is also worth noting how these ERPs relate to previous behavioral findings (Cohn and Wittenberg [<reflink idref="bib18" id="ref170">18</reflink>]). In prior work, action stars had an average viewing time of ~ 650 ms, and blank panels (comparable in function to noise panels) had an average viewing time of ~ 750 ms. This means participants' self-paced responses to advance to a subsequent panel would have just followed the processing of the N400 at these panels. Later effects (P600, LFP) would have therefore spilled over to the subsequent panel in self-paced viewing, perhaps contributing to the slower viewing times at that next position, even for action stars in sequences with scrambled panels. Nevertheless, spillover effects may not fully account for the increase in viewing times here, as we did indeed find ERP effects at the panels following action stars and noise panels.</p> <p>At the panel after the critical manipulation, in the 500–800 and 800–1100 ms epochs we observed a sustained negativity with a rightward posterior distribution that was larger to panels after action stars and noise panels than event-explicit panels. While this finding implies additional differential processing between explicit and non-explicit panels, the nature of such processing is not clear by the distribution and polarity of the effect. The timing and posterior distribution is consistent with the P600 (Brouwer et al. [<reflink idref="bib6" id="ref171">6</reflink>]; Leckey and Federmeier [<reflink idref="bib38" id="ref172">38</reflink>]; Van Petten and Luka [<reflink idref="bib67" id="ref173">67</reflink>]), yet a P600 to panels after explicit events would be the opposite of what would be predicted. Alternatively, later negativities would be consistent with proposals of a stage of interpretive semantic processing (Baggio [<reflink idref="bib1" id="ref174">1</reflink>]). Sustained negativities with a frontal distribution in sentence and discourse processing have been thought to index working memory processes such as those operating to build inferred event information (Baggio et al. [<reflink idref="bib2" id="ref175">2</reflink>]; Bott [<reflink idref="bib5" id="ref176">5</reflink>]; Paczynski et al. [<reflink idref="bib56" id="ref177">56</reflink>]; Wittenberg et al. [<reflink idref="bib70" id="ref178">70</reflink>]), or search processes linking anaphors to referential information (Hoeks and Brouwer [<reflink idref="bib28" id="ref179">28</reflink>]; van Berkum [<reflink idref="bib65" id="ref180">65</reflink>]). Sustained negativities have appeared to panels in visual narratives following an inference (Cohn and Kutas [<reflink idref="bib14" id="ref181">14</reflink>]), like at the action star, but in an earlier epoch and a more anterior distribution. Here, we observed negativities with a slightly later latency and more posterior distribution to panels that were posited to create inferences.</p> <p>An interpretation of these negativities indexing sustained working memory processes would make sense for the inference needed following a panel with impoverished event information, like action stars or noise panels. This would be consistent with proposals that sustained negativities index processes of holding information in memory to resolve ambiguities for a mental model in discourse (Baggio [<reflink idref="bib1" id="ref182">1</reflink>]; van Berkum [<reflink idref="bib65" id="ref183">65</reflink>]). This ERP pattern is consistent with behavioral findings of slower self-paced viewing times for panels following an inference, including action stars, compared to those after explicit events (Cohn and Wittenberg [<reflink idref="bib18" id="ref184">18</reflink>]; Huff et al. [<reflink idref="bib30" id="ref185">30</reflink>]; Hutson et al. [<reflink idref="bib31" id="ref186">31</reflink>]; Magliano et al. [<reflink idref="bib45" id="ref187">45</reflink>]). One such study introduced working memory load tasks between images prior to participants reaching the inference-generating panel (Magliano et al. [<reflink idref="bib45" id="ref188">45</reflink>]), which interfered with bridging inference generation. Interestingly, both verbal and visual working memory load tasks affected the processing of the <emph>visual</emph> narrative, implying a domain-general process. Nevertheless, if this sustained negativity indexes such a process, it is unclear why it has a posterior distribution here, rather than the more frontal distribution observed in prior work in both verbal and visual domains, and at the prior action star panel. Such disparity requires further work for clarity.</p> <p>Finally, the lack of a difference between these negativities between panels following action stars and noise panels suggests both panels triggered similar processes. This is consistent with findings that self-paced viewing times did not differ for panels after action stars compared to blank panels (Cohn and Wittenberg [<reflink idref="bib18" id="ref189">18</reflink>]), which could imply similar attempts to reconcile the absence of meaningful information across these inferential techniques. In line with this, we found no difference between participants' ratings of how much these sequences "made sense," despite several participants recognizing the difference between action stars and noise panels in unprompted responses to post-hoc questionnaires. Thus, these panels only seemed to differ in the later effects to the panels themselves (action stars: sustained negativity/P600, noise panels: LFP), perhaps suggesting only that they departed only in recognition of their conventionality. Overall, these findings suggest that the absence of key information in a sequence may result in similar processing and assessment of their coherence, despite differences between inferential techniques themselves.</p> <hd id="AN0148720207-14">Conclusions</hd> <p>This study examined the ERPs for panels which substitute for explicit events in visual narratives, thereby sponsoring bridging inferences. We observed cascading processing mechanisms similar to those observed in studies of language processing, which varied depending on the conventional (action stars) or less conventional (noise panels) properties of these panels. However, the absence of depicted events in both seemed to sponsor similar attempts to reconcile this missing information. Altogether, these findings provide an initial look at the neurocognition of inference generation across sequential images, further demonstrating that inference must balance the implicit of what is omitted and the explicit of what is provided, even when what is shown remains inexplicit.</p> <hd id="AN0148720207-15">Acknowledgements</hd> <p>Maaike Verest is thanked for assistance with data collection, and Emily Coderre and Tom Foulsham for assistance with pre-test norming of stimuli.</p> <hd id="AN0148720207-16">Authors' contributions</hd> <p>NC conceived of and carried out the experiments, analyses, and writing of the final manuscript. The author read and approved the final manuscript.</p> <hd id="AN0148720207-17">Funding</hd> <p>This study received no funding.</p> <hd id="AN0148720207-18">Availability of data and materials</hd> <p>The dataset used and/or analysed during the current study are available from the corresponding author on reasonable request.</p> <hd id="AN0148720207-19">Ethics approval and consent to participate</hd> <p>This study was approved by the Research Ethics Committee of Tilburg School of Humanities and Digital Sciences at Tilburg University (Identification code: REC#:2018/17). All participants gave informed written consent prior to the experiment.</p> <hd id="AN0148720207-20">Consent for publication</hd> <p>Not applicable.</p> <hd id="AN0148720207-21">Competing interests</hd> <p>The author declares that he has no competing interests.</p> <hd id="AN0148720207-22">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0148720207-23"> <title> References </title> <blist> <bibl id="bib1" idref="ref43" type="bt">1</bibl> <bibtext> Baggio G. Meaning in the brain. 2018: Cambridge, MA; MIT Press. 10.7551/mitpress/11265.001.0001</bibtext> </blist> <blist> <bibl id="bib2" idref="ref83" type="bt">2</bibl> <bibtext> Baggio G, van Lambalgen M, Hagoort P. Computing and recomputing discourse models: An ERP study. Journal of Memory and Language. 2008; 59; 1: 36-53. 10.1016/j.jml.2008.02.005</bibtext> </blist> <blist> <bibl id="bib3" idref="ref78" type="bt">3</bibl> <bibtext> Bornkessel-Schlesewsky I, Schlesewsky M. An alternative perspective on "semantic P600" effects in language comprehension. Brain Research Reviews. 2008; 59; 1: 55-73. 10.1016/j.brainresrev.2008.05.003. 18617270</bibtext> </blist> <blist> <bibl id="bib4" idref="ref153" type="bt">4</bibl> <bibtext> Bornkessel-Schlesewsky I, Schlesewsky M. Toward a neurobiologically plausible model of language-related, negative event-related potentials. Frontiers in Psychology. 2019; 10; 298: 1-17. 10.3389/fpsyg.2019.00298</bibtext> </blist> <blist> <bibl id="bib5" idref="ref84" type="bt">5</bibl> <bibtext> Bott O. The processing of events. 2010: Amsterdam; John Benjamins Publishing Company. 10.1075/la.162</bibtext> </blist> <blist> <bibl id="bib6" idref="ref67" type="bt">6</bibl> <bibtext> Brouwer H, Crocker MW, Venhuizen NJ, Hoeks JCJ. A neurocomputational model of the N400 and the P600 in language processing. Cognitive Science. 2016; 41; S6: 1318-1352. 10.1111/cogs.12461. 28000963. 5484319</bibtext> </blist> <blist> <bibl id="bib7" idref="ref48" type="bt">7</bibl> <bibtext> Coderre EL, O'Donnell E, O'Rourke E, Cohn N. Predictability modulates the N400 in non-verbal narrative processing. Scientific Reports. 2020; 10: 10326. 10.1038/s41598-020-66814-z. 32587312. 7316725</bibtext> </blist> <blist> <bibl id="bib8" idref="ref21" type="bt">8</bibl> <bibtext> Cohn N. The visual language of comics: Introduction to the structure and cognition of sequential images. 2013: London; Bloomsbury</bibtext> </blist> <blist> <bibl id="bib9" idref="ref29" type="bt">9</bibl> <bibtext> Cohn N. You're a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science. 2014; 38; 7: 1317-1359. 10.1111/cogs.12116. 24646175. 4159446</bibtext> </blist> <blist> <bibtext> Cohn N. Being explicit about the implicit: Inference generating techniques in visual narrative. Language and Cognition. 2019; 11; 1: 66-97. 10.1017/langcog.2019.6</bibtext> </blist> <blist> <bibtext> Cohn N. Your brain on comics: A cognitive model of visual narrative comprehension. Topics in Cognitive Science. 2020; 12; 1: 352-386. 10.1111/tops.12421. 30963724</bibtext> </blist> <blist> <bibtext> Cohn N, Foulsham T. Zooming in on the cognitive neuroscience of visual narrative. Brain and Cognition. 2020; 146: 105634. 10.1016/j.bandc.2020.105634. 33157490</bibtext> </blist> <blist> <bibtext> Cohn N, Jackendoff R, Holcomb PJ, Kuperberg GR. The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension. Neuropsychologia. 2014; 64: 63-70. 10.1016/j.neuropsychologia.2014.09.018. 25241329. 4364919</bibtext> </blist> <blist> <bibtext> Cohn N, Kutas M. Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. Neuropsychologia. 2015; 77: 267-278. 10.1016/j.neuropsychologia.2015.08.026. 26320706. 4609637</bibtext> </blist> <blist> <bibtext> Cohn N, Kutas M. What's your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive Research: Principles and Implications. 2017; 2; 27: 1-13. 10.1186/s41235-017-0064-5</bibtext> </blist> <blist> <bibtext> Cohn N, Maher S. The notion of the motion: The neurocognition of motion lines in visual narratives. Brain Research. 2015; 1601: 73-84. 10.1016/j.brainres.2015.01.018. 25601006. 4376351</bibtext> </blist> <blist> <bibtext> Cohn N, Paczynski M, Jackendoff R, Holcomb PJ, Kuperberg GR. (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive Psychology. 2012; 65; 1: 1-38. 10.1016/j.cogpsych.2012.01.003. 22387723. 3331971</bibtext> </blist> <blist> <bibtext> Cohn N, Wittenberg E. Action starring narratives and events: Structure and inference in visual narrative comprehension. Journal of Cognitive Psychology. 2015; 27; 7: 812-828. 10.1080/20445911.2015.1051535. 26709362</bibtext> </blist> <blist> <bibtext> Donchin E, Coles MGH. Is the P300 component a manifestation of context updating?. Behavioral and Brain Sciences. 1988; 11; 03: 357-374. 10.1017/S0140525X00058027</bibtext> </blist> <blist> <bibtext> Draschkow D, Heikel E, Võ MLH, Fiebach CJ, Sassenhagen J. No evidence from MVPA for different processes underlying the N300 and N400 incongruity effects in object-scene processing. Neuropsychologia. 2018; 120: 9-17. 10.1016/j.neuropsychologia.2018.09.016. 30261162</bibtext> </blist> <blist> <bibtext> Foulsham T, Cohn N. Zooming in on visual narrative comprehension. Memory & Cognition. 2020. 10.3758/s13421-020-01101-w</bibtext> </blist> <blist> <bibtext> Foulsham T, Wybrow D, Cohn N. Reading without words: Eye movements in the comprehension of comic strips. Applied Cognitive Psychology. 2016; 30: 566-579. 10.1002/acp.3229</bibtext> </blist> <blist> <bibtext> Gernsbacher MA, Varner KR, Faust M. Investigating differences in general comprehension skill. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1990; 16: 430-445. 2140402</bibtext> </blist> <blist> <bibtext> Hagoort P. The core and beyond in the language-ready brain. Neuroscience & Biobehavioral Reviews. 2017; 81: 194-204. 10.1016/j.neubiorev.2017.01.048</bibtext> </blist> <blist> <bibtext> Hagoort P, Brown CM, Groothusen J. The syntactic positive shift (SPS) as an ERP measure of syntactic processing. Language and Cognitive Processes. 1993; 8; 4: 439-483. 10.1080/01690969308407585</bibtext> </blist> <blist> <bibtext> Hamm JP, Johnson BW, Kirk IJ. Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts. Clinical Neurophysiology. 2002; 113; 8: 1339-1350. 10.1016/S1388-2457(02)00161-X. 12140015</bibtext> </blist> <blist> <bibtext> Herman DBrône G, Vandaele J. Cognitive approaches to narrative analysis. Cognitive poetics: Goals, gains and gaps. 2009: New York; Walter de Gruyter: 30-43</bibtext> </blist> <blist> <bibtext> Hoeks JCJ, Brouwer HHoltgraves TM. Electrophysiological research on conversation and discourse. The Oxford Handbook of Language and Social Psychology. 2014: Oxford, UK; Oxford University Press: 365-386</bibtext> </blist> <blist> <bibtext> Huff M, Meitz TGK, Papenmeier F. Changes in situation models modulate processes of event perception in audiovisual narratives. Journal of Experimental Psychology: Learning, Memory, & Cognition. 2014; 40; 5: 1377-1388. 10.1037/0033-2909.133.2.273</bibtext> </blist> <blist> <bibtext> Huff M, Rosenfelder D, Oberbeck M, Merkt M, Papenmeier F, Meitz TGK. Cross-codal integration of bridging-event information in narrative understanding. Memory & Cognition. 2020. 10.3758/s13421-020-01039-z</bibtext> </blist> <blist> <bibtext> Hutson JP, Magliano J, Loschky LC. Understanding moment-to-moment processing of visual narratives. Cognitive Science. 2018; 42; 8: 2999-3033. 10.1111/cogs.12699. 30447018. 6587724</bibtext> </blist> <blist> <bibtext> Kendeou P, McMaster KL, Butterfuss R, Kim J, Bresina B, Wagner K. The inferential language comprehension (iLC) framework: Supporting children's comprehension of visual narratives. Topics in Cognitive Science. 2020; 12; 1: 256-273. 10.1111/tops.12457</bibtext> </blist> <blist> <bibtext> Kuperberg GR. Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events. Language, Cognition and Neuroscience. 2016; 31; 5: 602-616. 10.1080/23273798.2015.1130233. 27570786. 4996121</bibtext> </blist> <blist> <bibtext> Kuperberg GR, Paczynski M, Ditman T. Establishing causal coherence across sentences: An ERP study. Journal of Cognitive Neuroscience. 2011; 23; 5: 1230-1246. 10.1162/jocn.2010.21452</bibtext> </blist> <blist> <bibtext> Kutas M, Federmeier KD. Thirty years and counting: Finding meaning in the N400 component of the Event-Related Brain Potential (ERP). Annual Review of Psychology. 2011; 62; 1: 621-647. 10.1146/annurev.psych.093008.131123. 20809790. 4052444</bibtext> </blist> <blist> <bibtext> Kutas M, Hillyard SA. Reading senseless sentences: Brain potential reflect semantic incongruity. Science. 1980; 207: 203-205. 10.1126/science.7350657</bibtext> </blist> <blist> <bibtext> Laubrock J, Hohenstein S, Kümmerer MDunst A, Laubrock J, Wildfeuer J. Attention to comics: Cognitive processing during the reading of graphic literature. Empirical comics research: Digital, multimodal, and cognitive methods. 2018: New York; Routledge: 239-263. 10.4324/9781315185354-12</bibtext> </blist> <blist> <bibtext> Leckey M, Federmeier KD. The P3b and P600(s): Positive contributions to language comprehension. Psychophysiology. 2019. 10.1111/psyp.13351. 30802979</bibtext> </blist> <blist> <bibtext> Lopez-Calderon J, Luck SJ. ERPLAB: An open-source toolbox for the analysis of event-related potentials. Frontiers in Human Neuroscience. 2014; 8: 213. 10.3389/fnhum.2014.00213. 24782741. 3995046</bibtext> </blist> <blist> <bibtext> Loschky LC, Hutson JP, Smith ME, Smith TJ, Magliano JDunst A, Laubrock J, Wildfeuer J. Viewing static visual narratives through the lens of the scene perception and event comprehension theory (SPECT). Empirical comics research: Digital, multimodal, and cognitive methods. 2018: London; Routledge: 217-238. 10.4324/9781315185354-11</bibtext> </blist> <blist> <bibtext> Loschky LC, Magliano J, Larson AM, Smith TJ. The scene perception & event comprehension theory (SPECT) applied to visual narratives. Topics in Cognitive Science. 2020; 12; 1: 311-351. 10.1111/tops.12455. 31486277</bibtext> </blist> <blist> <bibtext> Magliano JP, Dijkstra K, Zwaan RA. Generating predictive inferences while viewing a movie. Discourse Processes. 1996; 22: 199-224. 10.1080/01638539609544973</bibtext> </blist> <blist> <bibtext> Magliano JP, Higgs K, Clinton JAGrishakova M, Poulaki M. Sources of complexity in comprehension across modalities of narrative experience. Narrative complexity: Cognition, embodiment, evolution. 2019: Lincoln; University of Nebraska Press: 149-173. 10.2307/j.ctvhktjh6.12</bibtext> </blist> <blist> <bibtext> Magliano JP, Kopp K, Higgs K, Rapp DN. Filling in the Gaps: Memory implications for inferring missing content in graphic narratives. Discourse Processes. 2017; 54; 8: 569-582. 10.1080/0163853X.2015.1136870</bibtext> </blist> <blist> <bibtext> Magliano JP, Larson AM, Higgs K, Loschky LC. The relative roles of visuospatial and linguistic working memory systems in generating inferences during visual narrative comprehension. Memory & Cognition. 2015; 44; 2: 207-219. 10.3758/s13421-015-0558-7</bibtext> </blist> <blist> <bibtext> Magliano JP, Zacks JM. The impact of continuity editing in narrative film on event segmentation. Cognitive Science. 2011; 35; 8: 1489-1517. 10.1111/j.1551-6709.2011.01202.x. 21972849. 3208769</bibtext> </blist> <blist> <bibtext> Manfredi M, Cohn N, Kutas M. When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative. Brain and Language. 2017; 169: 28-38. 10.1016/j.bandl.2017.02.001. 28242517. 5465314</bibtext> </blist> <blist> <bibtext> McCloud S. Understanding comics: The invisible art. 1993: New York, NY; Harper Collins</bibtext> </blist> <blist> <bibtext> McKoon G, Ratcliff R. Inference during reading. Psychological Review. 1992; 99; 3: 440-466. 10.1037/0033-295X.99.3.440</bibtext> </blist> <blist> <bibtext> McNamara DS, Magliano J. Toward a comprehensive model of comprehension. Psychology of learning and motivation. 2009; 51: 297-384. 10.1016/S0079-7421(09)51009-2</bibtext> </blist> <blist> <bibtext> McPherson WB, Holcomb PJ. An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology. 1999; 36; 1: 53-65. 10.1017/S0048577299971196</bibtext> </blist> <blist> <bibtext> Michalon O, Baggio G. Meaning-driven syntactic predictions in a parallel processing architecture: Theory and algorithmic modeling of ERP effects. Neuropsychologia. 2019; 131: 171-183. 10.1016/j.neuropsychologia.2019.05.009. 31100344</bibtext> </blist> <blist> <bibtext> Neville HJ, Nicol JL, Barss A, Forster KI, Garrett MF. Syntactically based sentence processing classes: Evidence from event-related brain potentials. Journal of Cognitive Neuroscience. 1991; 3; 2: 151-165. 10.1162/jocn.1991.3.2.151</bibtext> </blist> <blist> <bibtext> Nieuwland MS, Barr DJ, Bartolozzi F, Busch-Moreno S, Darley E, Donaldson DI. Dissociable effects of prediction and integration during language comprehension: Evidence from a large-scale study using brain potentials. Philosophical Transactions of the Royal Society B: Biological Sciences. 2020; 375; 1791: 20180522. 10.1098/rstb.2018.0522</bibtext> </blist> <blist> <bibtext> Osterhout L, Holcomb P. Event-related potentials elicited by syntactic anomaly. Journal of Memory and Language. 1992; 31: 758-806. 10.1016/0749-596X(92)90039-Z</bibtext> </blist> <blist> <bibtext> Paczynski M, Jackendoff R, Kuperberg G. When events change their nature: The neurocognitive mechanisms underlying aspectual coercion. Journal of Cognitive Neuroscience. 2014; 26; 9: 1905-1917. 10.1162/jocn_a_00638. 24702457. 4204801</bibtext> </blist> <blist> <bibtext> Peirce J, Gray JR, Simpson S, MacAskill M, Höchenberger R, Sogo H. PsychoPy2: Experiments in behavior made easy. Behavior Research Methods. 2019; 51; 1: 195-203. 10.3758/s13428-018-01193-y. 30734206. 6420413</bibtext> </blist> <blist> <bibtext> Polich J. Updating P300: An integrative theory of P3a and P3b. Clinical Neurophysiology. 2007; 118; 10: 2128-2148. 10.1016/j.clinph.2007.04.019</bibtext> </blist> <blist> <bibtext> Pratha NK, Avunjian N, Cohn N. Pow, punch, pika, and chu: The structure of sound effects in genres of American comics and Japanese manga. Multimodal Communication. 2016; 5; 2: 93-109. 10.1515/mc-2016-0017</bibtext> </blist> <blist> <bibtext> Rossion B, Caharel S. ERP evidence for the speed of face categorization in the human brain: Disentangling the contribution of low-level visual cues from face perception. Vision Research. 2011; 51; 12: 1297-1311. 10.1016/j.visres.2011.04.003</bibtext> </blist> <blist> <bibtext> Saraceni MCohn N. Relatedness: Aspects of textual connectivity in comics. The visual narrative reader. 2016: London; Bloomsbury: 115-129</bibtext> </blist> <blist> <bibtext> Sitnikova T, Holcomb PJ, Kuperberg GR. Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. Journal of Cognitive Neuroscience. 2008; 20; 11: 1-21. 10.1162/jocn.2008.20143</bibtext> </blist> <blist> <bibtext> St. George M, Mannes S, Hoffman JE. Individual differences in inference generation: An ERP analysis. Journal of Cognitive Neuroscience. 1997; 9; 6: 776-787. 10.1162/jocn.1997.9.6.776</bibtext> </blist> <blist> <bibtext> Taylor MJ, McCarthy G, Saliba E, Degiovanni E. ERP evidence of developmental changes in processing of faces. Clinical Neurophysiology. 1999; 110; 5: 910-915. 10.1016/S1388-2457(99)00006-1</bibtext> </blist> <blist> <bibtext> van Berkum JJASauerland U, Yatsushiro K. The neuropragmatics of "simple" utterance comprehension: An ERP review. Semantics and pragmatics: From experiment to theory. 2009: Basingstoke; Palgrave: 276-316</bibtext> </blist> <blist> <bibtext> van Dijk T, Kintsch W. Strategies of discourse comprehension. 1983: New York; Academic Press</bibtext> </blist> <blist> <bibtext> Van Petten C, Luka BJ. Prediction during language comprehension: Benefits, costs, and ERP components. International Journal of Psychophysiology. 2012; 83; 2: 176-190. 10.1016/j.ijpsycho.2011.09.015. 22019481</bibtext> </blist> <blist> <bibtext> Võ ML-H, Wolfe JM. Differential electrophysiological signatures of semantic and syntactic scene processing. Psychological Science. 2013; 24; 9: 1816-1823. 10.1177/0956797613476955. 23842954. 4838599</bibtext> </blist> <blist> <bibtext> West WC, Holcomb P. Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research. 2002; 13: 363-375. 10.1016/S0926-6410(01)00129-X</bibtext> </blist> <blist> <bibtext> Wittenberg E, Paczynski M, Wiese H, Jackendoff R, Kuperberg G. The difference between "giving a rose" and "giving a kiss": Sustained neural activity to the light verb construction. Journal of Memory and Language. 2014; 73: 31-42. 10.1016/j.jml.2014.02.002. 24910498. 4045490</bibtext> </blist> <blist> <bibtext> Zwaan RARoss BH. The immersed experiencer: Toward an embodied theory of language comprehension. The psychology of learning and motivation. 2004: New York; Academic Press: 35-62</bibtext> </blist> <blist> <bibtext> Zwaan RA, Radvansky GA. Situation models in language comprehension and memory. Psychological Bulletin. 1998; 123; 2: 162-185. 10.1037/0033-2909.123.2.162</bibtext> </blist> </ref> <aug> <p>By Neil Cohn</p> <p>Reported by Author</p> </aug> <nolink nlid="nl1" bibid="bib50" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib11" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib41" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib48" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib61" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib23" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib43" firstref="ref8"></nolink> <nolink nlid="nl8" bibid="bib32" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib49" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib30" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib31" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib45" firstref="ref18"></nolink> <nolink nlid="nl13" bibid="bib44" firstref="ref19"></nolink> <nolink nlid="nl14" bibid="bib10" firstref="ref22"></nolink> <nolink nlid="nl15" bibid="bib47" firstref="ref23"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref24"></nolink> <nolink nlid="nl17" bibid="bib71" firstref="ref26"></nolink> <nolink nlid="nl18" bibid="bib18" firstref="ref31"></nolink> <nolink nlid="nl19" bibid="bib40" firstref="ref33"></nolink> <nolink nlid="nl20" bibid="bib22" firstref="ref34"></nolink> <nolink nlid="nl21" bibid="bib21" firstref="ref37"></nolink> <nolink nlid="nl22" bibid="bib37" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib35" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib36" firstref="ref42"></nolink> <nolink nlid="nl25" bibid="bib24" firstref="ref44"></nolink> <nolink nlid="nl26" bibid="bib33" firstref="ref45"></nolink> <nolink nlid="nl27" bibid="bib54" firstref="ref47"></nolink> <nolink nlid="nl28" bibid="bib69" firstref="ref49"></nolink> <nolink nlid="nl29" bibid="bib34" firstref="ref50"></nolink> <nolink nlid="nl30" bibid="bib63" firstref="ref51"></nolink> <nolink nlid="nl31" bibid="bib51" firstref="ref52"></nolink> <nolink nlid="nl32" bibid="bib26" firstref="ref53"></nolink> <nolink nlid="nl33" bibid="bib20" firstref="ref54"></nolink> <nolink nlid="nl34" bibid="bib68" firstref="ref56"></nolink> <nolink nlid="nl35" bibid="bib66" firstref="ref58"></nolink> <nolink nlid="nl36" bibid="bib72" firstref="ref59"></nolink> <nolink nlid="nl37" bibid="bib29" firstref="ref60"></nolink> <nolink nlid="nl38" bibid="bib46" firstref="ref61"></nolink> <nolink nlid="nl39" bibid="bib67" firstref="ref63"></nolink> <nolink nlid="nl40" bibid="bib25" firstref="ref64"></nolink> <nolink nlid="nl41" bibid="bib55" firstref="ref65"></nolink> <nolink nlid="nl42" bibid="bib19" firstref="ref69"></nolink> <nolink nlid="nl43" bibid="bib38" firstref="ref70"></nolink> <nolink nlid="nl44" bibid="bib12" firstref="ref72"></nolink> <nolink nlid="nl45" bibid="bib14" firstref="ref73"></nolink> <nolink nlid="nl46" bibid="bib13" firstref="ref75"></nolink> <nolink nlid="nl47" bibid="bib52" firstref="ref80"></nolink> <nolink nlid="nl48" bibid="bib28" firstref="ref85"></nolink> <nolink nlid="nl49" bibid="bib65" firstref="ref86"></nolink> <nolink nlid="nl50" bibid="bib17" firstref="ref88"></nolink> <nolink nlid="nl51" bibid="bib16" firstref="ref108"></nolink> <nolink nlid="nl52" bibid="bib57" firstref="ref111"></nolink> <nolink nlid="nl53" bibid="bib15" firstref="ref114"></nolink> <nolink nlid="nl54" bibid="bib39" firstref="ref116"></nolink> <nolink nlid="nl55" bibid="bib42" firstref="ref130"></nolink> <nolink nlid="nl56" bibid="bib60" firstref="ref133"></nolink> <nolink nlid="nl57" bibid="bib64" firstref="ref134"></nolink> <nolink nlid="nl58" bibid="bib62" firstref="ref138"></nolink> <nolink nlid="nl59" bibid="bib53" firstref="ref154"></nolink> <nolink nlid="nl60" bibid="bib59" firstref="ref161"></nolink> <nolink nlid="nl61" bibid="bib58" firstref="ref168"></nolink> <nolink nlid="nl62" bibid="bib56" firstref="ref177"></nolink> <nolink nlid="nl63" bibid="bib70" firstref="ref178"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1287610
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Starring Role for Inference in the Neurocognition of Visual Narratives
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Cohn%2C+Neil%22">Cohn, Neil</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-0074-1995">0000-0003-0074-1995</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Cognitive+Research%3A+Principles+and+Implications%22"><i>Cognitive Research: Principles and Implications</i></searchLink>. 2021 6.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 14
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2021
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Inferences%22">Inferences</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Brain%22">Brain</searchLink><br /><searchLink fieldCode="DE" term="%22Visual+Learning%22">Visual Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Cartoons%22">Cartoons</searchLink><br /><searchLink fieldCode="DE" term="%22Schemata+%28Cognition%29%22">Schemata (Cognition)</searchLink><br /><searchLink fieldCode="DE" term="%22Brain+Hemisphere+Functions%22">Brain Hemisphere Functions</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Processing%22">Language Processing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1186/s41235-021-00270-9
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2365-7464
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped "action stars" where a reader "knows" events are undepicted "at that moment," rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a "noise" panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300-500 ms), but action stars and noise panels then differed in their later effects (500-900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2021
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1287610
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1287610
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1186/s41235-021-00270-9
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 14
    Subjects:
      – SubjectFull: Inferences
        Type: general
      – SubjectFull: Cognitive Processes
        Type: general
      – SubjectFull: Brain
        Type: general
      – SubjectFull: Visual Learning
        Type: general
      – SubjectFull: Cartoons
        Type: general
      – SubjectFull: Schemata (Cognition)
        Type: general
      – SubjectFull: Brain Hemisphere Functions
        Type: general
      – SubjectFull: Language Processing
        Type: general
    Titles:
      – TitleFull: A Starring Role for Inference in the Neurocognition of Visual Narratives
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Cohn, Neil
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-electronic
              Value: 2365-7464
          Numbering:
            – Type: volume
              Value: 6
          Titles:
            – TitleFull: Cognitive Research: Principles and Implications
              Type: main
ResultId 1