View in EDS

Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity

Saved in:
Bibliographic Details
Title: Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity
Language: English
Authors: Joshua B. Gilbert (ORCID 0000-0003-3496-2710), Luke W. Miratrix (ORCID 0000-0002-0078-1906), Mridul Joshi, Benjamin W. Domingue (ORCID 0000-0002-3894-9049)
Source: Journal of Educational and Behavioral Statistics. 2025 50(1):72-101.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 30
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Early Childhood Education
Grade 2
Primary Education
Descriptors: Causal Models, Item Response Theory, Statistical Inference, Psychometrics, Educational Assessment, Randomized Controlled Trials, Reading Instruction, Intervention, Elementary School Students, Grade 2, Reading Tests, Pretesting, Correlation, Effect Size, Test Items, Item Analysis, Difficulty Level, Test Construction
DOI: 10.3102/10769986241240085
ISSN: 1076-9986
1935-1054
Abstract: Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment. This study demonstrates that the identical patterns of HTE on test score outcomes can emerge either from variation in treatment effects due to a preintervention participant characteristic or from correlations between treatment effects and item easiness parameters. We demonstrate analytically and through simulation that these two scenarios cannot be distinguished if analysis is based on summary scores alone. We then describe a novel approach that identifies the relevant data-generating process by leveraging item-level data. We apply our approach to a randomized trial of a reading intervention in second grade and show that any apparent HTE by pretest ability is driven by the correlation between treatment effect size and item easiness. Our results highlight the potential of employing measurement principles in causal analysis, beyond their common use in test construction.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1457138
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1457138
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Joshua+B%2E+Gilbert%22">Joshua B. Gilbert</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3496-2710">0000-0003-3496-2710</externalLink>)<br /><searchLink fieldCode="AR" term="%22Luke+W%2E+Miratrix%22">Luke W. Miratrix</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0078-1906">0000-0002-0078-1906</externalLink>)<br /><searchLink fieldCode="AR" term="%22Mridul+Joshi%22">Mridul Joshi</searchLink><br /><searchLink fieldCode="AR" term="%22Benjamin+W%2E+Domingue%22">Benjamin W. Domingue</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-3894-9049">0000-0002-3894-9049</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+and+Behavioral+Statistics%22"><i>Journal of Educational and Behavioral Statistics</i></searchLink>. 2025 50(1):72-101.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 30
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Early+Childhood+Education%22">Early Childhood Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+2%22">Grade 2</searchLink><br /><searchLink fieldCode="EL" term="%22Primary+Education%22">Primary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Causal+Models%22">Causal Models</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Inference%22">Statistical Inference</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Assessment%22">Educational Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Randomized+Controlled+Trials%22">Randomized Controlled Trials</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Instruction%22">Reading Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+2%22">Grade 2</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Tests%22">Reading Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Pretesting%22">Pretesting</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Effect+Size%22">Effect Size</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.3102/10769986241240085
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1076-9986<br />1935-1054
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment. This study demonstrates that the identical patterns of HTE on test score outcomes can emerge either from variation in treatment effects due to a preintervention participant characteristic or from correlations between treatment effects and item easiness parameters. We demonstrate analytically and through simulation that these two scenarios cannot be distinguished if analysis is based on summary scores alone. We then describe a novel approach that identifies the relevant data-generating process by leveraging item-level data. We apply our approach to a randomized trial of a reading intervention in second grade and show that any apparent HTE by pretest ability is driven by the correlation between treatment effect size and item easiness. Our results highlight the potential of employing measurement principles in causal analysis, beyond their common use in test construction.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1457138
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1457138
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3102/10769986241240085
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 30
        StartPage: 72
    Subjects:
      – SubjectFull: Causal Models
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Statistical Inference
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Educational Assessment
        Type: general
      – SubjectFull: Randomized Controlled Trials
        Type: general
      – SubjectFull: Reading Instruction
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Grade 2
        Type: general
      – SubjectFull: Reading Tests
        Type: general
      – SubjectFull: Pretesting
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Effect Size
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Test Construction
        Type: general
    Titles:
      – TitleFull: Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Joshua B. Gilbert
      – PersonEntity:
          Name:
            NameFull: Luke W. Miratrix
      – PersonEntity:
          Name:
            NameFull: Mridul Joshi
      – PersonEntity:
          Name:
            NameFull: Benjamin W. Domingue
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1076-9986
            – Type: issn-electronic
              Value: 1935-1054
          Numbering:
            – Type: volume
              Value: 50
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Educational and Behavioral Statistics
              Type: main
ResultId 1