View in EDS

Detecting the Impact of Remote Proctored At-Home Testing Using Propensity Score Weighting. Research Report. ETS RR-24-11

Saved in:
Bibliographic Details
Title: Detecting the Impact of Remote Proctored At-Home Testing Using Propensity Score Weighting. Research Report. ETS RR-24-11
Language: English
Authors: Jing Miao, Yi Cao, Michael E. Walker
Source: ETS Research Report Series. Dec 2024.
Availability: ETS. Rosedale Road, Mailstop 19R, Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
Peer Reviewed: Y
Page Count: 34
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Testing, Test Format, Computer Assisted Testing, Home Study, Distance Education, Weighted Scores, Student Diversity, Comparative Testing
ISSN: 2330-8516
Abstract: Studies of test score comparability have been conducted at different stages in the history of testing to ensure that test results carry the same meaning regardless of test conditions. The expansion of at-home testing via remote proctoring sparked another round of interest. This study uses data from three licensure tests to assess potential mode effects associated with the dual option of on-site testing at test centers and at-home testing via remote proctoring. We generated propensity score weights to balance the two self-selected groups in order to detect the mode effect on the test outcomes. We also assessed the potential impact of omitted variables on the estimated mode effect. Results of the study indicate that the demographic compositions of the test takers are similar before and after the introduction of the RP option. Examinees under the two testing modes differ slightly on certain background variables. Once the group differences are adjusted by propensity score weighting, the estimated mode effects are small and nonsystematic across test titles overall. We note some variations across subgroups based on gender and race.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1459574
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1459574
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1459574
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Detecting the Impact of Remote Proctored At-Home Testing Using Propensity Score Weighting. Research Report. ETS RR-24-11
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Jing+Miao%22">Jing Miao</searchLink><br /><searchLink fieldCode="AR" term="%22Yi+Cao%22">Yi Cao</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+E%2E+Walker%22">Michael E. Walker</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22ETS+Research+Report+Series%22"><i>ETS Research Report Series</i></searchLink>. Dec 2024.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: ETS. Rosedale Road, Mailstop 19R, Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 34
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Home+Study%22">Home Study</searchLink><br /><searchLink fieldCode="DE" term="%22Distance+Education%22">Distance Education</searchLink><br /><searchLink fieldCode="DE" term="%22Weighted+Scores%22">Weighted Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Diversity%22">Student Diversity</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Testing%22">Comparative Testing</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2330-8516
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Studies of test score comparability have been conducted at different stages in the history of testing to ensure that test results carry the same meaning regardless of test conditions. The expansion of at-home testing via remote proctoring sparked another round of interest. This study uses data from three licensure tests to assess potential mode effects associated with the dual option of on-site testing at test centers and at-home testing via remote proctoring. We generated propensity score weights to balance the two self-selected groups in order to detect the mode effect on the test outcomes. We also assessed the potential impact of omitted variables on the estimated mode effect. Results of the study indicate that the demographic compositions of the test takers are similar before and after the introduction of the RP option. Examinees under the two testing modes differ slightly on certain background variables. Once the group differences are adjusted by propensity score weighting, the estimated mode effects are small and nonsystematic across test titles overall. We note some variations across subgroups based on gender and race.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1459574
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1459574
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 34
    Subjects:
      – SubjectFull: Testing
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Home Study
        Type: general
      – SubjectFull: Distance Education
        Type: general
      – SubjectFull: Weighted Scores
        Type: general
      – SubjectFull: Student Diversity
        Type: general
      – SubjectFull: Comparative Testing
        Type: general
    Titles:
      – TitleFull: Detecting the Impact of Remote Proctored At-Home Testing Using Propensity Score Weighting. Research Report. ETS RR-24-11
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Jing Miao
      – PersonEntity:
          Name:
            NameFull: Yi Cao
      – PersonEntity:
          Name:
            NameFull: Michael E. Walker
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-electronic
              Value: 2330-8516
          Titles:
            – TitleFull: ETS Research Report Series
              Type: main
ResultId 1