View in EDS

The Precision and Bias of Cut Score Estimates from the Beuk Standard Setting Method

Saved in:
Bibliographic Details
Title: The Precision and Bias of Cut Score Estimates from the Beuk Standard Setting Method
Language: English
Authors: Joseph H. Grochowalski (ORCID 0000-0002-9617-2346), Lei Wan, Lauren Molin, Amy H. Hendrickson (ORCID 0009-0003-0199-0640)
Source: Journal of Educational Measurement. 2025 62(4):687-717.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 31
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Cutting Scores, Standard Setting, Accuracy, Statistical Bias, Simulation, Interrater Reliability
DOI: 10.1111/jedm.70007
ISSN: 0022-0655
1745-3984
Abstract: The Beuk standard setting method derives cut scores through expert judgment that balances content and normative perspectives. This study developed a method to estimate confidence intervals for Beuk settings and assessed their accuracy via simulations. Simulations varied SME panel size, expert agreement, cut score locations, score distributions, and decision alignment. Panels with 20+ participants provided precise and accurate cut score estimates if strongly agreed upon. Larger panels did not improve precision significantly. Cut score location influenced confidence interval widths, highlighting its importance in planning. Real data showed SME disagreement increased bias and variance of Beuk estimates. Use Beuk cut scores cautiously with small panels, flat score distributions, or significant expert disagreement.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1491384
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1491384
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Precision and Bias of Cut Score Estimates from the Beuk Standard Setting Method
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Joseph+H%2E+Grochowalski%22">Joseph H. Grochowalski</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9617-2346">0000-0002-9617-2346</externalLink>)<br /><searchLink fieldCode="AR" term="%22Lei+Wan%22">Lei Wan</searchLink><br /><searchLink fieldCode="AR" term="%22Lauren+Molin%22">Lauren Molin</searchLink><br /><searchLink fieldCode="AR" term="%22Amy+H%2E+Hendrickson%22">Amy H. Hendrickson</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0003-0199-0640">0009-0003-0199-0640</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+Measurement%22"><i>Journal of Educational Measurement</i></searchLink>. 2025 62(4):687-717.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 31
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Cutting+Scores%22">Cutting Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Standard+Setting%22">Standard Setting</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Bias%22">Statistical Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Simulation%22">Simulation</searchLink><br /><searchLink fieldCode="DE" term="%22Interrater+Reliability%22">Interrater Reliability</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jedm.70007
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0655<br />1745-3984
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The Beuk standard setting method derives cut scores through expert judgment that balances content and normative perspectives. This study developed a method to estimate confidence intervals for Beuk settings and assessed their accuracy via simulations. Simulations varied SME panel size, expert agreement, cut score locations, score distributions, and decision alignment. Panels with 20+ participants provided precise and accurate cut score estimates if strongly agreed upon. Larger panels did not improve precision significantly. Cut score location influenced confidence interval widths, highlighting its importance in planning. Real data showed SME disagreement increased bias and variance of Beuk estimates. Use Beuk cut scores cautiously with small panels, flat score distributions, or significant expert disagreement.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1491384
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1491384
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jedm.70007
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 31
        StartPage: 687
    Subjects:
      – SubjectFull: Cutting Scores
        Type: general
      – SubjectFull: Standard Setting
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Statistical Bias
        Type: general
      – SubjectFull: Simulation
        Type: general
      – SubjectFull: Interrater Reliability
        Type: general
    Titles:
      – TitleFull: The Precision and Bias of Cut Score Estimates from the Beuk Standard Setting Method
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Joseph H. Grochowalski
      – PersonEntity:
          Name:
            NameFull: Lei Wan
      – PersonEntity:
          Name:
            NameFull: Lauren Molin
      – PersonEntity:
          Name:
            NameFull: Amy H. Hendrickson
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0022-0655
            – Type: issn-electronic
              Value: 1745-3984
          Numbering:
            – Type: volume
              Value: 62
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Journal of Educational Measurement
              Type: main
ResultId 1