View in EDS

Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses. EdWorkingPaper No. 25-1303

Saved in:
Bibliographic Details
Title: Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses. EdWorkingPaper No. 25-1303
Language: English
Authors: Lily An, Edward J. Kim, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 21
Publication Date: 2025
Document Type: Reports - Research
Descriptors: Test Bias, Statistical Analysis, Intersectionality, Sampling, Statistical Inference
Abstract: Modern test developers conduct differential item functioning (DIF) analyses to ensure fairness in educational and psychological testing. To address previously unrecognized biases, researchers have recently demonstrated the importance of conducting intersectional DIF analyses that attend to the intersectional nature of test-takers' multiple identities. However, these intersectional DIF approaches overlook how overlapping identity categories affect the statistical validity of DIF analyses. As the related tests violate independence, typical p-value corrections used in intersectional DIF analyses such as Bonferroni yield overly conservative family wise error rates (FWER) which limit statistical power to identify true DIF. Additionally, DIF on one dimension can spuriously cause DIF to appear while testing another demographic dimension with high overlap, a phenomenon we call signal interference. These concerns are particularly aggravated in intersectional DIF. We offer an approach utilizing parametric bootstrapping that adjusts significance levels of DIF detection processes to yield the intended Type I error rates. Using simulations studies, we illustrate the statistical complications of intersectional DIF analyses and the ability of our proposed method to resolve them.
Abstractor: As Provided
Entry Date: 2026
Accession Number: ED678279
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED678279
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED678279
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses. EdWorkingPaper No. 25-1303
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lily+An%22">Lily An</searchLink><br /><searchLink fieldCode="AR" term="%22Edward+J%2E+Kim%22">Edward J. Kim</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 21
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Bias%22">Test Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Intersectionality%22">Intersectionality</searchLink><br /><searchLink fieldCode="DE" term="%22Sampling%22">Sampling</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Inference%22">Statistical Inference</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Modern test developers conduct differential item functioning (DIF) analyses to ensure fairness in educational and psychological testing. To address previously unrecognized biases, researchers have recently demonstrated the importance of conducting intersectional DIF analyses that attend to the intersectional nature of test-takers' multiple identities. However, these intersectional DIF approaches overlook how overlapping identity categories affect the statistical validity of DIF analyses. As the related tests violate independence, typical p-value corrections used in intersectional DIF analyses such as Bonferroni yield overly conservative family wise error rates (FWER) which limit statistical power to identify true DIF. Additionally, DIF on one dimension can spuriously cause DIF to appear while testing another demographic dimension with high overlap, a phenomenon we call signal interference. These concerns are particularly aggravated in intersectional DIF. We offer an approach utilizing parametric bootstrapping that adjusts significance levels of DIF detection processes to yield the intended Type I error rates. Using simulations studies, we illustrate the statistical complications of intersectional DIF analyses and the ability of our proposed method to resolve them.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED678279
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED678279
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 21
    Subjects:
      – SubjectFull: Test Bias
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Intersectionality
        Type: general
      – SubjectFull: Sampling
        Type: general
      – SubjectFull: Statistical Inference
        Type: general
    Titles:
      – TitleFull: Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses. EdWorkingPaper No. 25-1303
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Lily An
      – PersonEntity:
          Name:
            NameFull: Edward J. Kim
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1