View in EDS PDF Full Text

Effects of Visual Speech Envelope on Audiovisual Speech Perception in Multitalker Listening Environments.

Saved in:
Bibliographic Details
Title: Effects of Visual Speech Envelope on Audiovisual Speech Perception in Multitalker Listening Environments.
Authors: Yi Yuan1, Meyers, Kelli1, Borges, Kayla1, Lleo, Yasneli1, Fiorentino, Katarina A.1, Yonghee Oh1 yoh@phhp.ufl.edu
Source: Journal of Speech, Language & Hearing Research. Jul2021, Vol. 64 Issue 7, p2845-2853. 9p. 2 Charts, 3 Graphs.
Subject Terms: *Speech perception, *Audiovisual materials, Noise
Abstract: Purpose: This study investigated the effects of visually presented speech envelope information with various modulation rates and depths on audiovisual speech perception in noise. Method: Forty adults (21.25 ± 1.45 years) participated in audiovisual sentence recognition measurements in noise. Target speech sentences were auditorily presented in multitalker babble noises at a -3dBSNR.Acoustic amplitude envelopes of target signals were extracted through low-pass filters with different cutoff frequencies (4, 10, and 30 Hz) and a fixed modulation depth at 100% (Experiment 1) or extracted with various modulation depths (0%, 25%, 50%, 75%, and 100%) and a fixed 10-Hz modulation rate (Experiment 2). The extracted target envelopes were synchronized with the amplitude of a spherical-shaped ball and presented as visual stimuli. Subjects were instructed to attend to both auditory and visual stimuli of the target sentences and type down their answers. The sentence recognition accuracy was compared between audio-only and audiovisual conditions. Results: In Experiment 1, a significant improvement in speech intelligibility was observed when the visual analog (a sphere) synced with the acoustic amplitude envelope modulated at a 10-Hz modulation rate compared to the audio-only condition. In Experiment 2, the visual analog with 75% modulation depth resulted in better audiovisual speech perception in noise compared to the other modulation depth conditions. Conclusion: An abstract visual analog of acoustic amplitude envelopes can be efficiently delivered by the visual system and integrated online with auditory signals to enhance speech perception in noise, independent of particular articulation movements. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 151481531
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Effects of Visual Speech Envelope on Audiovisual Speech Perception in Multitalker Listening Environments.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Yi+Yuan%22">Yi Yuan</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Meyers%2C+Kelli%22">Meyers, Kelli</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Borges%2C+Kayla%22">Borges, Kayla</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Lleo%2C+Yasneli%22">Lleo, Yasneli</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Fiorentino%2C+Katarina+A%2E%22">Fiorentino, Katarina A.</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Yonghee+Oh%22">Yonghee Oh</searchLink><relatesTo>1</relatesTo><i> yoh@phhp.ufl.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Speech%2C+Language+%26+Hearing+Research%22">Journal of Speech, Language & Hearing Research</searchLink>. Jul2021, Vol. 64 Issue 7, p2845-2853. 9p. 2 Charts, 3 Graphs.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Speech+perception%22">Speech perception</searchLink><br />*<searchLink fieldCode="DE" term="%22Audiovisual+materials%22">Audiovisual materials</searchLink><br /><searchLink fieldCode="DE" term="%22Noise%22">Noise</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Purpose: This study investigated the effects of visually presented speech envelope information with various modulation rates and depths on audiovisual speech perception in noise. Method: Forty adults (21.25 ± 1.45 years) participated in audiovisual sentence recognition measurements in noise. Target speech sentences were auditorily presented in multitalker babble noises at a -3dBSNR.Acoustic amplitude envelopes of target signals were extracted through low-pass filters with different cutoff frequencies (4, 10, and 30 Hz) and a fixed modulation depth at 100% (Experiment 1) or extracted with various modulation depths (0%, 25%, 50%, 75%, and 100%) and a fixed 10-Hz modulation rate (Experiment 2). The extracted target envelopes were synchronized with the amplitude of a spherical-shaped ball and presented as visual stimuli. Subjects were instructed to attend to both auditory and visual stimuli of the target sentences and type down their answers. The sentence recognition accuracy was compared between audio-only and audiovisual conditions. Results: In Experiment 1, a significant improvement in speech intelligibility was observed when the visual analog (a sphere) synced with the acoustic amplitude envelope modulated at a 10-Hz modulation rate compared to the audio-only condition. In Experiment 2, the visual analog with 75% modulation depth resulted in better audiovisual speech perception in noise compared to the other modulation depth conditions. Conclusion: An abstract visual analog of acoustic amplitude envelopes can be efficiently delivered by the visual system and integrated online with auditory signals to enhance speech perception in noise, independent of particular articulation movements. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=151481531
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1044/2021_JSLHR-20-00688
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 9
        StartPage: 2845
    Subjects:
      – SubjectFull: Speech perception
        Type: general
      – SubjectFull: Audiovisual materials
        Type: general
      – SubjectFull: Noise
        Type: general
    Titles:
      – TitleFull: Effects of Visual Speech Envelope on Audiovisual Speech Perception in Multitalker Listening Environments.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yi Yuan
      – PersonEntity:
          Name:
            NameFull: Meyers, Kelli
      – PersonEntity:
          Name:
            NameFull: Borges, Kayla
      – PersonEntity:
          Name:
            NameFull: Lleo, Yasneli
      – PersonEntity:
          Name:
            NameFull: Fiorentino, Katarina A.
      – PersonEntity:
          Name:
            NameFull: Yonghee Oh
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Text: Jul2021
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-print
              Value: 10924388
          Numbering:
            – Type: volume
              Value: 64
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: Journal of Speech, Language & Hearing Research
              Type: main
ResultId 1