View in EDS

Performance of Automated Speech Scoring on Different Low- to Medium-Entropy Item Types for Low-Proficiency English Learners. Research Report. ETS RR-17-12

Saved in:
Bibliographic Details
Title: Performance of Automated Speech Scoring on Different Low- to Medium-Entropy Item Types for Low-Proficiency English Learners. Research Report. ETS RR-17-12
Language: English
Authors: Loukina, Anastassia, Zechner, Klaus, Yoon, Su-Youn, Zhang, Mo, Tao, Jidong, Wang, Xinhao, Lee, Chong Min, Mulholland, Matthew
Source: ETS Research Report Series. Mar 2017.
Availability: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Peer Reviewed: Y
Page Count: 19
Publication Date: 2017
Document Type: Journal Articles
Reports - Research
Descriptors: Automation, Scoring, Speech Tests, Test Items, Language Proficiency, English (Second Language), Second Language Learning, Test Scoring Machines, Models, Comparative Analysis
ISSN: 2330-8516
Abstract: This report presents an overview of the "SpeechRater"? automated scoring engine model building and evaluation process for several item types with a focus on a low-English-proficiency test-taker population. We discuss each stage of speech scoring, including automatic speech recognition, filtering models for nonscorable responses, and scoring model building and evaluation and compare how the performance at each step differs between different item types. We conclude by discussing the effect of item type on automated scoring performance. We also give recommendations about what considerations should be taken into account when developing tests for low-proficiency English speakers to obtain reliable scores from an automatic scoring engine.
Abstractor: As Provided
Number of References: 22
Entry Date: 2018
Accession Number: EJ1168911
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1168911
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1168911
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Performance of Automated Speech Scoring on Different Low- to Medium-Entropy Item Types for Low-Proficiency English Learners. Research Report. ETS RR-17-12
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Loukina%2C+Anastassia%22">Loukina, Anastassia</searchLink><br /><searchLink fieldCode="AR" term="%22Zechner%2C+Klaus%22">Zechner, Klaus</searchLink><br /><searchLink fieldCode="AR" term="%22Yoon%2C+Su-Youn%22">Yoon, Su-Youn</searchLink><br /><searchLink fieldCode="AR" term="%22Zhang%2C+Mo%22">Zhang, Mo</searchLink><br /><searchLink fieldCode="AR" term="%22Tao%2C+Jidong%22">Tao, Jidong</searchLink><br /><searchLink fieldCode="AR" term="%22Wang%2C+Xinhao%22">Wang, Xinhao</searchLink><br /><searchLink fieldCode="AR" term="%22Lee%2C+Chong+Min%22">Lee, Chong Min</searchLink><br /><searchLink fieldCode="AR" term="%22Mulholland%2C+Matthew%22">Mulholland, Matthew</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22ETS+Research+Report+Series%22"><i>ETS Research Report Series</i></searchLink>. Mar 2017.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 19
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2017
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Speech+Tests%22">Speech Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Proficiency%22">Language Proficiency</searchLink><br /><searchLink fieldCode="DE" term="%22English+%28Second+Language%29%22">English (Second Language)</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Learning%22">Second Language Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Scoring+Machines%22">Test Scoring Machines</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2330-8516
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This report presents an overview of the "SpeechRater"? automated scoring engine model building and evaluation process for several item types with a focus on a low-English-proficiency test-taker population. We discuss each stage of speech scoring, including automatic speech recognition, filtering models for nonscorable responses, and scoring model building and evaluation and compare how the performance at each step differs between different item types. We conclude by discussing the effect of item type on automated scoring performance. We also give recommendations about what considerations should be taken into account when developing tests for low-proficiency English speakers to obtain reliable scores from an automatic scoring engine.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 22
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2018
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1168911
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1168911
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 19
    Subjects:
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Speech Tests
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Language Proficiency
        Type: general
      – SubjectFull: English (Second Language)
        Type: general
      – SubjectFull: Second Language Learning
        Type: general
      – SubjectFull: Test Scoring Machines
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
    Titles:
      – TitleFull: Performance of Automated Speech Scoring on Different Low- to Medium-Entropy Item Types for Low-Proficiency English Learners. Research Report. ETS RR-17-12
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Loukina, Anastassia
      – PersonEntity:
          Name:
            NameFull: Zechner, Klaus
      – PersonEntity:
          Name:
            NameFull: Yoon, Su-Youn
      – PersonEntity:
          Name:
            NameFull: Zhang, Mo
      – PersonEntity:
          Name:
            NameFull: Tao, Jidong
      – PersonEntity:
          Name:
            NameFull: Wang, Xinhao
      – PersonEntity:
          Name:
            NameFull: Lee, Chong Min
      – PersonEntity:
          Name:
            NameFull: Mulholland, Matthew
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2017
          Identifiers:
            – Type: issn-electronic
              Value: 2330-8516
          Titles:
            – TitleFull: ETS Research Report Series
              Type: main
ResultId 1