Calibration of Automatically Generated Items Using Bayesian Hierarchical Modeling.
Saved in:
| Title: | Calibration of Automatically Generated Items Using Bayesian Hierarchical Modeling. |
|---|---|
| Language: | English |
| Authors: | Johnson, Matthew S., Sinharay, Sandip |
| Peer Reviewed: | N |
| Page Count: | 32 |
| Publication Date: | 2003 |
| Document Type: | Reports - Descriptive |
| Descriptors: | Bayesian Statistics, Constructed Response, Educational Assessment, Estimation (Mathematics), Markov Processes, Monte Carlo Methods, Multiple Choice Tests, Test Items |
| Abstract: | For complex educational assessments, there is an increasing use of "item families," which are groups of related items. However, calibration or scoring for such an assessment requires fitting models that take into account the dependence structure inherent among the items that belong to the same item family. C. Glas and W. van der Linden (2001) suggest a Bayesian hierarchical model to analyze data involving item families with multiple choice items. This paper extends the model to take into account item families with constructed response items, and designs a Markov chain Monte Carlo algorithm for the Bayesian estimation of model parameters. The hierarchical model, which accounts for the dependence structure inherent among the items, implicitly defines the "family response function" (FRF) for the score categories. This paper suggests a way to combine the FRFs over the score categories to obtain a "family score function" (FSF), which is a quick graphical summary of the expected score of an individual with a certain ability to an item randomly generated from an item family. The paper also suggests a method for the Bayesian estimation of the FRF and FSF. This work is a significant step towards building a tool to analyze data involving item families, and it may be very useful practically, for example, in automatic item generation systems that create tests involving item families. (Contains 1 table, 10 figures, and 25 references.) (Author/SLD) |
| Entry Date: | 2003 |
| Accession Number: | ED476471 |
| Database: | ERIC |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED476471 Name: ERIC Full Text Category: fullText Text: Full Text from ERIC |
|---|---|
| Header | DbId: eric DbLabel: ERIC An: ED476471 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Calibration of Automatically Generated Items Using Bayesian Hierarchical Modeling. – Name: Language Label: Language Group: Lang Data: English – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Johnson%2C+Matthew+S%2E%22">Johnson, Matthew S.</searchLink><br /><searchLink fieldCode="AR" term="%22Sinharay%2C+Sandip%22">Sinharay, Sandip</searchLink> – Name: PeerReviewed Label: Peer Reviewed Group: SrcInfo Data: N – Name: Pages Label: Page Count Group: Src Data: 32 – Name: DatePubCY Label: Publication Date Group: Date Data: 2003 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Reports - Descriptive – Name: Subject Label: Descriptors Group: Su Data: <searchLink fieldCode="DE" term="%22Bayesian+Statistics%22">Bayesian Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Constructed+Response%22">Constructed Response</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Assessment%22">Educational Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+%28Mathematics%29%22">Estimation (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Markov+Processes%22">Markov Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Monte+Carlo+Methods%22">Monte Carlo Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: For complex educational assessments, there is an increasing use of "item families," which are groups of related items. However, calibration or scoring for such an assessment requires fitting models that take into account the dependence structure inherent among the items that belong to the same item family. C. Glas and W. van der Linden (2001) suggest a Bayesian hierarchical model to analyze data involving item families with multiple choice items. This paper extends the model to take into account item families with constructed response items, and designs a Markov chain Monte Carlo algorithm for the Bayesian estimation of model parameters. The hierarchical model, which accounts for the dependence structure inherent among the items, implicitly defines the "family response function" (FRF) for the score categories. This paper suggests a way to combine the FRFs over the score categories to obtain a "family score function" (FSF), which is a quick graphical summary of the expected score of an individual with a certain ability to an item randomly generated from an item family. The paper also suggests a method for the Bayesian estimation of the FRF and FSF. This work is a significant step towards building a tool to analyze data involving item families, and it may be very useful practically, for example, in automatic item generation systems that create tests involving item families. (Contains 1 table, 10 figures, and 25 references.) (Author/SLD) – Name: DateEntry Label: Entry Date Group: Date Data: 2003 – Name: AN Label: Accession Number Group: ID Data: ED476471 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED476471 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English PhysicalDescription: Pagination: PageCount: 32 Subjects: – SubjectFull: Bayesian Statistics Type: general – SubjectFull: Constructed Response Type: general – SubjectFull: Educational Assessment Type: general – SubjectFull: Estimation (Mathematics) Type: general – SubjectFull: Markov Processes Type: general – SubjectFull: Monte Carlo Methods Type: general – SubjectFull: Multiple Choice Tests Type: general – SubjectFull: Test Items Type: general Titles: – TitleFull: Calibration of Automatically Generated Items Using Bayesian Hierarchical Modeling. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Johnson, Matthew S. – PersonEntity: Name: NameFull: Sinharay, Sandip IsPartOfRelationships: – BibEntity: Dates: – D: 12 M: 05 Type: published Y: 2003 |
| ResultId | 1 |
