View in EDS

Considerations for Designing Measures of Confidence

Saved in:
Bibliographic Details
Title: Considerations for Designing Measures of Confidence
Language: English
Authors: Casandra Koevoets-Beach (ORCID 0009-0004-1560-6934), Donya Kurdi, Morgan Balabanoff (ORCID 0000-0002-9197-8228)
Source: Practical Assessment, Research & Evaluation. 2025 30(1).
Availability: University of Massachusetts Amherst Libraries. 154 Hicks Way, Amherst, MA 01003. e-mail: pare@umass.edu; Web site: https://openpublishing.library.umass.edu/pare/
Peer Reviewed: Y
Page Count: 25
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Confidence Testing, Interviews, Metacognition, Undergraduate Students, Test Items, Self Evaluation (Individuals)
ISSN: 1531-7714
Abstract: Confidence tiers have been paired with multiple choice items across different fields since the early twentieth century and have seen widespread adoption in discipline-based education research fields seeking to evaluate aspects of self-regulated learning. The design of two-tiered confidence judgments impacts interpretability and perception of their utility, so meaningful engagement hinges on intentional design for specific constructs. This study uses cognitive interviews exploring students' interpretations of confidence tier components to identify design features which elicit meaningful variation in self-reflection. The evidence supports two prevailing motivations for using confidence tiers--prompting metacognition and measuring the strength of learners' alternate conceptions. The challenges and strategies students encounter while engaging with confidence tiers serve as the foundation to improve the validity of collected confidence data. Recommendations are presented to improve the clarity and utility of confidence tiers to provide meaningful evaluation of target constructs.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1491693
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1491693
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1491693
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Considerations for Designing Measures of Confidence
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Casandra+Koevoets-Beach%22">Casandra Koevoets-Beach</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0004-1560-6934">0009-0004-1560-6934</externalLink>)<br /><searchLink fieldCode="AR" term="%22Donya+Kurdi%22">Donya Kurdi</searchLink><br /><searchLink fieldCode="AR" term="%22Morgan+Balabanoff%22">Morgan Balabanoff</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9197-8228">0000-0002-9197-8228</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Practical+Assessment%2C+Research+%26+Evaluation%22"><i>Practical Assessment, Research & Evaluation</i></searchLink>. 2025 30(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: University of Massachusetts Amherst Libraries. 154 Hicks Way, Amherst, MA 01003. e-mail: pare@umass.edu; Web site: https://openpublishing.library.umass.edu/pare/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 25
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Confidence+Testing%22">Confidence Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Interviews%22">Interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Metacognition%22">Metacognition</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Self+Evaluation+%28Individuals%29%22">Self Evaluation (Individuals)</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1531-7714
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Confidence tiers have been paired with multiple choice items across different fields since the early twentieth century and have seen widespread adoption in discipline-based education research fields seeking to evaluate aspects of self-regulated learning. The design of two-tiered confidence judgments impacts interpretability and perception of their utility, so meaningful engagement hinges on intentional design for specific constructs. This study uses cognitive interviews exploring students' interpretations of confidence tier components to identify design features which elicit meaningful variation in self-reflection. The evidence supports two prevailing motivations for using confidence tiers--prompting metacognition and measuring the strength of learners' alternate conceptions. The challenges and strategies students encounter while engaging with confidence tiers serve as the foundation to improve the validity of collected confidence data. Recommendations are presented to improve the clarity and utility of confidence tiers to provide meaningful evaluation of target constructs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1491693
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1491693
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
    Subjects:
      – SubjectFull: Confidence Testing
        Type: general
      – SubjectFull: Interviews
        Type: general
      – SubjectFull: Metacognition
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Self Evaluation (Individuals)
        Type: general
    Titles:
      – TitleFull: Considerations for Designing Measures of Confidence
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Casandra Koevoets-Beach
      – PersonEntity:
          Name:
            NameFull: Donya Kurdi
      – PersonEntity:
          Name:
            NameFull: Morgan Balabanoff
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 1531-7714
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Practical Assessment, Research & Evaluation
              Type: main
ResultId 1