View in EDS

Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109

Saved in:
Bibliographic Details
Title: Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109
Language: English
Authors: Kenneth A. Shores, Sanford R. Student, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2024.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 68
Publication Date: 2024
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305S210008
Document Type: Reports - Research
Education Level: High Schools
Secondary Education
Higher Education
Postsecondary Education
Descriptors: High School Seniors, Academic Records, Course Selection (Students), Grades (Scholastic), Difficulty Level, Grade Point Average, College Entrance Examinations, Test Bias, Admission Criteria, Open Enrollment, Grade Inflation
Geographic Terms: Delaware
Assessment and Survey Identifiers: SAT (College Admission Test)
Abstract: We use student-level administrative data from Delaware for 43,767 high school students across five 12th grade cohorts from 2017 to 2021. We apply Item Response Theory (IRT) to high school transcript data, treating courses as items and grades as ordered responses, to estimate both student transcript strength ([theta]) and course difficulty. We prove, via construct and predictive validation and simulation, that [theta] improves upon GPA because it accounts for ability selection into courses with variable difficulty. Compared to the SAT, [theta] shows smaller racial/ethnic gaps but substantially larger gender gaps that indicate boys underperform in their courses relative to their standardized test scores. We conclude by discussing significant methodological--such as grade-inflation and cross-school heterogeneity in course offerings--and practical challenges that remain before such measures could be considered for high-stakes applications.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2025
Accession Number: ED671123
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED671123
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED671123
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kenneth+A%2E+Shores%22">Kenneth A. Shores</searchLink><br /><searchLink fieldCode="AR" term="%22Sanford+R%2E+Student%22">Sanford R. Student</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2024.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 68
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305S210008
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22High+School+Seniors%22">High School Seniors</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Records%22">Academic Records</searchLink><br /><searchLink fieldCode="DE" term="%22Course+Selection+%28Students%29%22">Course Selection (Students)</searchLink><br /><searchLink fieldCode="DE" term="%22Grades+%28Scholastic%29%22">Grades (Scholastic)</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+Point+Average%22">Grade Point Average</searchLink><br /><searchLink fieldCode="DE" term="%22College+Entrance+Examinations%22">College Entrance Examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Bias%22">Test Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Admission+Criteria%22">Admission Criteria</searchLink><br /><searchLink fieldCode="DE" term="%22Open+Enrollment%22">Open Enrollment</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+Inflation%22">Grade Inflation</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Delaware%22">Delaware</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22SAT+%28College+Admission+Test%29%22">SAT (College Admission Test)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We use student-level administrative data from Delaware for 43,767 high school students across five 12th grade cohorts from 2017 to 2021. We apply Item Response Theory (IRT) to high school transcript data, treating courses as items and grades as ordered responses, to estimate both student transcript strength ([theta]) and course difficulty. We prove, via construct and predictive validation and simulation, that [theta] improves upon GPA because it accounts for ability selection into courses with variable difficulty. Compared to the SAT, [theta] shows smaller racial/ethnic gaps but substantially larger gender gaps that indicate boys underperform in their courses relative to their standardized test scores. We conclude by discussing significant methodological--such as grade-inflation and cross-school heterogeneity in course offerings--and practical challenges that remain before such measures could be considered for high-stakes applications.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED671123
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED671123
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 68
    Subjects:
      – SubjectFull: High School Seniors
        Type: general
      – SubjectFull: Academic Records
        Type: general
      – SubjectFull: Course Selection (Students)
        Type: general
      – SubjectFull: Grades (Scholastic)
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Grade Point Average
        Type: general
      – SubjectFull: College Entrance Examinations
        Type: general
      – SubjectFull: Test Bias
        Type: general
      – SubjectFull: Admission Criteria
        Type: general
      – SubjectFull: Open Enrollment
        Type: general
      – SubjectFull: Grade Inflation
        Type: general
      – SubjectFull: Delaware
        Type: general
      – SubjectFull: SAT (College Admission Test)
        Type: general
    Titles:
      – TitleFull: Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Kenneth A. Shores
      – PersonEntity:
          Name:
            NameFull: Sanford R. Student
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1