View in EDS

A Topic Testlet Model for Calibrating Testlet Constructed Responses

Saved in:
Bibliographic Details
Title: A Topic Testlet Model for Calibrating Testlet Constructed Responses
Language: English
Authors: Jiawei Xiong (ORCID 0000-0002-2069-8720), Huan Kuang (ORCID 0000-0003-2651-2867), Cheng Tang (ORCID 0009-0004-6556-7144), Qidi Liu (ORCID 0000-0002-6797-4163), Bowen Wang (ORCID 0009-0001-5668-278X), George Engelhard (ORCID 0000-0002-1694-8942), Allan S. Cohen (ORCID 0000-0002-8776-9378), Xinhui Xiong, Rufei Sheng
Source: Journal of Educational Measurement. 2026 63(1).
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 40
Publication Date: 2026
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Secondary Education
Descriptors: Test Construction, Test Items, Item Analysis, Psychometrics, Item Response Theory, Goodness of Fit, Test Interpretation, Accuracy, Test Validity, Scores, Language Arts, Sciences, Elementary Secondary Education
DOI: 10.1111/jedm.70001
ISSN: 0022-0655
1745-3984
Abstract: Constructed responses (CRs) within testlets are widely used to assess complex skills but can pose calibration challenges due to local item dependence. A few current testlet models incorporate testlet-specific effects to address local dependence but struggle with interpreting these effects and may not fully capture the complexities of CR items because they rely only on response or score patterns. A Topic Testlet Model (TTM) integrates topic modeling within a psychometric framework was proposed. It uses latent topics from student written responses to adjust for local dependence, enable simultaneous calibration, and provide insights into evaluating student reasoning and writing in testlet CR items. Using empirical data from both English Language and Arts as well as Science assessments for grades 3-12, we compare the TTM with existing models in terms of ability estimates, item parameter estimates, and overall model fit. Simulation studies further demonstrate parameter recovery under various testing scenarios. Results show that the TTM effectively accounts for local dependence, improves testlet effect interpretability, and demonstrates a better fit than the existing models. TTM advances CR testlet calibration, leveraging additional information from student written responses to improve the precision of the assessment systems and validity of the use of test scores.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1501261
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1501261
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Topic Testlet Model for Calibrating Testlet Constructed Responses
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Jiawei+Xiong%22">Jiawei Xiong</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-2069-8720">0000-0002-2069-8720</externalLink>)<br /><searchLink fieldCode="AR" term="%22Huan+Kuang%22">Huan Kuang</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-2651-2867">0000-0003-2651-2867</externalLink>)<br /><searchLink fieldCode="AR" term="%22Cheng+Tang%22">Cheng Tang</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0004-6556-7144">0009-0004-6556-7144</externalLink>)<br /><searchLink fieldCode="AR" term="%22Qidi+Liu%22">Qidi Liu</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6797-4163">0000-0002-6797-4163</externalLink>)<br /><searchLink fieldCode="AR" term="%22Bowen+Wang%22">Bowen Wang</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0001-5668-278X">0009-0001-5668-278X</externalLink>)<br /><searchLink fieldCode="AR" term="%22George+Engelhard%22">George Engelhard</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-1694-8942">0000-0002-1694-8942</externalLink>)<br /><searchLink fieldCode="AR" term="%22Allan+S%2E+Cohen%22">Allan S. Cohen</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-8776-9378">0000-0002-8776-9378</externalLink>)<br /><searchLink fieldCode="AR" term="%22Xinhui+Xiong%22">Xinhui Xiong</searchLink><br /><searchLink fieldCode="AR" term="%22Rufei+Sheng%22">Rufei Sheng</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+Measurement%22"><i>Journal of Educational Measurement</i></searchLink>. 2026 63(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 40
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Psychometrics%22">Psychometrics</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Goodness+of+Fit%22">Goodness of Fit</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Interpretation%22">Test Interpretation</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Arts%22">Language Arts</searchLink><br /><searchLink fieldCode="DE" term="%22Sciences%22">Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jedm.70001
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0022-0655<br />1745-3984
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Constructed responses (CRs) within testlets are widely used to assess complex skills but can pose calibration challenges due to local item dependence. A few current testlet models incorporate testlet-specific effects to address local dependence but struggle with interpreting these effects and may not fully capture the complexities of CR items because they rely only on response or score patterns. A Topic Testlet Model (TTM) integrates topic modeling within a psychometric framework was proposed. It uses latent topics from student written responses to adjust for local dependence, enable simultaneous calibration, and provide insights into evaluating student reasoning and writing in testlet CR items. Using empirical data from both English Language and Arts as well as Science assessments for grades 3-12, we compare the TTM with existing models in terms of ability estimates, item parameter estimates, and overall model fit. Simulation studies further demonstrate parameter recovery under various testing scenarios. Results show that the TTM effectively accounts for local dependence, improves testlet effect interpretability, and demonstrates a better fit than the existing models. TTM advances CR testlet calibration, leveraging additional information from student written responses to improve the precision of the assessment systems and validity of the use of test scores.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1501261
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1501261
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jedm.70001
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 40
    Subjects:
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Psychometrics
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Goodness of Fit
        Type: general
      – SubjectFull: Test Interpretation
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Language Arts
        Type: general
      – SubjectFull: Sciences
        Type: general
      – SubjectFull: Elementary Secondary Education
        Type: general
    Titles:
      – TitleFull: A Topic Testlet Model for Calibrating Testlet Constructed Responses
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Jiawei Xiong
      – PersonEntity:
          Name:
            NameFull: Huan Kuang
      – PersonEntity:
          Name:
            NameFull: Cheng Tang
      – PersonEntity:
          Name:
            NameFull: Qidi Liu
      – PersonEntity:
          Name:
            NameFull: Bowen Wang
      – PersonEntity:
          Name:
            NameFull: George Engelhard
      – PersonEntity:
          Name:
            NameFull: Allan S. Cohen
      – PersonEntity:
          Name:
            NameFull: Xinhui Xiong
      – PersonEntity:
          Name:
            NameFull: Rufei Sheng
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 0022-0655
            – Type: issn-electronic
              Value: 1745-3984
          Numbering:
            – Type: volume
              Value: 63
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Educational Measurement
              Type: main
ResultId 1