The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments.
Saved in:
| Title: | The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments. |
|---|---|
| Authors: | König, Christoph1 (AUTHOR), Khorramdel, Lale2 (AUTHOR), Yamamoto, Kentaro2 (AUTHOR), Frey, Andreas1 (AUTHOR) |
| Source: | Educational Measurement: Issues & Practice. Spring2021, Vol. 40 Issue 1, p17-27. 11p. 2 Charts, 4 Graphs. |
| Subject Terms: | *Item response theory, Calibration, Sample size (Statistics), Countries, Small states |
| Company/Entity: | Programme for International Student Assessment |
| Abstract: | Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities. [ABSTRACT FROM AUTHOR] |
| Copyright of Educational Measurement: Issues & Practice is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 149246887 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22König%2C+Christoph%22">König, Christoph</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Khorramdel%2C+Lale%22">Khorramdel, Lale</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Yamamoto%2C+Kentaro%22">Yamamoto, Kentaro</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Frey%2C+Andreas%22">Frey, Andreas</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Educational+Measurement%3A+Issues+%26+Practice%22">Educational Measurement: Issues & Practice</searchLink>. Spring2021, Vol. 40 Issue 1, p17-27. 11p. 2 Charts, 4 Graphs. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Item+response+theory%22">Item response theory</searchLink><br /><searchLink fieldCode="DE" term="%22Calibration%22">Calibration</searchLink><br /><searchLink fieldCode="DE" term="%22Sample+size+%28Statistics%29%22">Sample size (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Countries%22">Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Small+states%22">Small states</searchLink> – Name: SubjectCompany Label: Company/Entity Group: Su Data: <searchLink fieldCode="DE" term="%22Programme+for+International+Student+Assessment%22">Programme for International Student Assessment</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Educational Measurement: Issues & Practice is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=149246887 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/emip.12381 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 11 StartPage: 17 Subjects: – SubjectFull: Item response theory Type: general – SubjectFull: Calibration Type: general – SubjectFull: Sample size (Statistics) Type: general – SubjectFull: Countries Type: general – SubjectFull: Small states Type: general – SubjectFull: Programme for International Student Assessment Type: general Titles: – TitleFull: The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: König, Christoph – PersonEntity: Name: NameFull: Khorramdel, Lale – PersonEntity: Name: NameFull: Yamamoto, Kentaro – PersonEntity: Name: NameFull: Frey, Andreas IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 03 Text: Spring2021 Type: published Y: 2021 Identifiers: – Type: issn-print Value: 07311745 Numbering: – Type: volume Value: 40 – Type: issue Value: 1 Titles: – TitleFull: Educational Measurement: Issues & Practice Type: main |
| ResultId | 1 |
