View in EDS

Exploring the Potential of a Video-Mediated Interactive Speaking Assessment. Research Report. ETS RR-19-05

Saved in:
Bibliographic Details
Title: Exploring the Potential of a Video-Mediated Interactive Speaking Assessment. Research Report. ETS RR-19-05
Language: English
Authors: Ockey, Gary J., Timpe-Laughlin, Veronika, Davis, Larry, Gu, Lin
Source: ETS Research Report Series. Dec 2019.
Availability: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Peer Reviewed: Y
Page Count: 29
Publication Date: 2019
Document Type: Journal Articles
Reports - Research
Tests/Questionnaires
Education Level: Higher Education
Postsecondary Education
Descriptors: Computer Assisted Testing, Speech Skills, Interpersonal Competence, Videoconferencing, Oral Language, Technology Integration, Instructional Effectiveness, Interaction, College Students, Student Satisfaction, Foreign Countries, English Language Learners, Usability
Geographic Terms: United States, China (Guangzhou), China (Shanghai)
ISSN: 2330-8516
Abstract: The construct of oral ability is multifaceted, but due to technological and practical constraints, the majority of computer-delivered speaking assessments are designed to measure only certain aspects of this ability. Most notably, interactional competence, which we define as the ability to actively structure dialogic speech in real time, is often not assessed. Innovations in technology, namely, computermediated video, make it possible for test takers in different locations to see and speak with others in real time and may make it achievable for computer-based tests to assess more aspects of oral communication, including interactional competence. This report gives the findings from a study that explored to what extent computer-mediated video, namely, Skype, could function in conjunction with a platform designed to present four innovative speaking tasks that could conceivably assess a broad construct of oral ability. The overarching goals of this project were twofold. First, we aimed to determine (a) the degree to which current computer video-mediated technology can be used effectively to deliver assessments remotely and (b) the extent to which participants felt that four specific tasks could assess speaking ability by means of this technology. The speaking tasks included giving short responses to an interlocutor's questions, summarizing a proposal, defending a position in a group discussion, and giving a prepared presentation and responding to questions from other participants. Two data collections were conducted: one with all 72 participants located in the United States and one with all 74 participants located in China. The findings provide preliminary evidence that the stability of computer video-mediated technology varies considerably, with technical disruptions being relatively few in the U.S. trial but very frequent in the China context. Moreover, the findings suggest that participants viewed the tasks as generally representing interactive speaking activities that they encounter in the oral language use domain, affording them the opportunity to demonstrate their oral abilities, and that these tasks can be effectively completed in a computer video-mediated environment when technology cooperates.
Abstractor: As Provided
Entry Date: 2020
Accession Number: EJ1238536
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1238536
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1238536
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Exploring the Potential of a Video-Mediated Interactive Speaking Assessment. Research Report. ETS RR-19-05
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ockey%2C+Gary+J%2E%22">Ockey, Gary J.</searchLink><br /><searchLink fieldCode="AR" term="%22Timpe-Laughlin%2C+Veronika%22">Timpe-Laughlin, Veronika</searchLink><br /><searchLink fieldCode="AR" term="%22Davis%2C+Larry%22">Davis, Larry</searchLink><br /><searchLink fieldCode="AR" term="%22Gu%2C+Lin%22">Gu, Lin</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22ETS+Research+Report+Series%22"><i>ETS Research Report Series</i></searchLink>. Dec 2019.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 29
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2019
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research<br />Tests/Questionnaires
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Speech+Skills%22">Speech Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Interpersonal+Competence%22">Interpersonal Competence</searchLink><br /><searchLink fieldCode="DE" term="%22Videoconferencing%22">Videoconferencing</searchLink><br /><searchLink fieldCode="DE" term="%22Oral+Language%22">Oral Language</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Integration%22">Technology Integration</searchLink><br /><searchLink fieldCode="DE" term="%22Instructional+Effectiveness%22">Instructional Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Interaction%22">Interaction</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Satisfaction%22">Student Satisfaction</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22English+Language+Learners%22">English Language Learners</searchLink><br /><searchLink fieldCode="DE" term="%22Usability%22">Usability</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+States%22">United States</searchLink><br /><searchLink fieldCode="DE" term="%22China+%28Guangzhou%29%22">China (Guangzhou)</searchLink><br /><searchLink fieldCode="DE" term="%22China+%28Shanghai%29%22">China (Shanghai)</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2330-8516
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The construct of oral ability is multifaceted, but due to technological and practical constraints, the majority of computer-delivered speaking assessments are designed to measure only certain aspects of this ability. Most notably, interactional competence, which we define as the ability to actively structure dialogic speech in real time, is often not assessed. Innovations in technology, namely, computermediated video, make it possible for test takers in different locations to see and speak with others in real time and may make it achievable for computer-based tests to assess more aspects of oral communication, including interactional competence. This report gives the findings from a study that explored to what extent computer-mediated video, namely, Skype, could function in conjunction with a platform designed to present four innovative speaking tasks that could conceivably assess a broad construct of oral ability. The overarching goals of this project were twofold. First, we aimed to determine (a) the degree to which current computer video-mediated technology can be used effectively to deliver assessments remotely and (b) the extent to which participants felt that four specific tasks could assess speaking ability by means of this technology. The speaking tasks included giving short responses to an interlocutor's questions, summarizing a proposal, defending a position in a group discussion, and giving a prepared presentation and responding to questions from other participants. Two data collections were conducted: one with all 72 participants located in the United States and one with all 74 participants located in China. The findings provide preliminary evidence that the stability of computer video-mediated technology varies considerably, with technical disruptions being relatively few in the U.S. trial but very frequent in the China context. Moreover, the findings suggest that participants viewed the tasks as generally representing interactive speaking activities that they encounter in the oral language use domain, affording them the opportunity to demonstrate their oral abilities, and that these tasks can be effectively completed in a computer video-mediated environment when technology cooperates.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2020
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1238536
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1238536
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 29
    Subjects:
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Speech Skills
        Type: general
      – SubjectFull: Interpersonal Competence
        Type: general
      – SubjectFull: Videoconferencing
        Type: general
      – SubjectFull: Oral Language
        Type: general
      – SubjectFull: Technology Integration
        Type: general
      – SubjectFull: Instructional Effectiveness
        Type: general
      – SubjectFull: Interaction
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Student Satisfaction
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: English Language Learners
        Type: general
      – SubjectFull: Usability
        Type: general
      – SubjectFull: United States
        Type: general
      – SubjectFull: China (Guangzhou)
        Type: general
      – SubjectFull: China (Shanghai)
        Type: general
    Titles:
      – TitleFull: Exploring the Potential of a Video-Mediated Interactive Speaking Assessment. Research Report. ETS RR-19-05
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ockey, Gary J.
      – PersonEntity:
          Name:
            NameFull: Timpe-Laughlin, Veronika
      – PersonEntity:
          Name:
            NameFull: Davis, Larry
      – PersonEntity:
          Name:
            NameFull: Gu, Lin
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2019
          Identifiers:
            – Type: issn-electronic
              Value: 2330-8516
          Titles:
            – TitleFull: ETS Research Report Series
              Type: main
ResultId 1