Writing evaluation: rater and task effects on the reliability of writing scores for children in Grades 3 and 4.
Saved in:
| Title: | Writing evaluation: rater and task effects on the reliability of writing scores for children in Grades 3 and 4. |
|---|---|
| Authors: | Kim, Young-Suk1 youngsk7@uci.edu, Schatschneider, Christopher2, Wanzek, Jeanne3, Gatlin, Brandy4, Al Otaiba, Stephanie5 |
| Source: | Reading & Writing. Jun2017, Vol. 30 Issue 6, p1287-1310. 24p. |
| Subject Terms: | *Composition (Language arts), *Dictation (Educational method), *Prewriting (Writing process), *Research papers (Students), Extinct languages |
| Abstract: | We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of .90 and .80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written compositions were evaluated in widely used evaluation methods for developing writers: holistic scoring, productivity, and curriculum-based writing scores. Results showed that 54 and 52% of variance in narrative and expository compositions were attributable to true individual differences in writing. Students' scores varied largely by tasks (30.44 and 28.61% of variance), but not by raters. To reach the reliability of .90, multiple tasks and raters were needed, and for the reliability of .80, a single rater and multiple tasks were needed. These findings offer important implications about reliably evaluating children's writing skills, given that writing is typically evaluated by a single task and a single rater in classrooms and even in some state accountability systems. [ABSTRACT FROM AUTHOR] |
| Copyright of Reading & Writing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 123224578 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Writing evaluation: rater and task effects on the reliability of writing scores for children in Grades 3 and 4. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Kim%2C+Young-Suk%22">Kim, Young-Suk</searchLink><relatesTo>1</relatesTo><i> youngsk7@uci.edu</i><br /><searchLink fieldCode="AR" term="%22Schatschneider%2C+Christopher%22">Schatschneider, Christopher</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Wanzek%2C+Jeanne%22">Wanzek, Jeanne</searchLink><relatesTo>3</relatesTo><br /><searchLink fieldCode="AR" term="%22Gatlin%2C+Brandy%22">Gatlin, Brandy</searchLink><relatesTo>4</relatesTo><br /><searchLink fieldCode="AR" term="%22Al+Otaiba%2C+Stephanie%22">Al Otaiba, Stephanie</searchLink><relatesTo>5</relatesTo> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Reading+%26+Writing%22">Reading & Writing</searchLink>. Jun2017, Vol. 30 Issue 6, p1287-1310. 24p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Composition+%28Language+arts%29%22">Composition (Language arts)</searchLink><br />*<searchLink fieldCode="DE" term="%22Dictation+%28Educational+method%29%22">Dictation (Educational method)</searchLink><br />*<searchLink fieldCode="DE" term="%22Prewriting+%28Writing+process%29%22">Prewriting (Writing process)</searchLink><br />*<searchLink fieldCode="DE" term="%22Research+papers+%28Students%29%22">Research papers (Students)</searchLink><br /><searchLink fieldCode="DE" term="%22Extinct+languages%22">Extinct languages</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: We examined how raters and tasks influence measurement error in writing evaluation and how many raters and tasks are needed to reach a desirable level of .90 and .80 reliabilities for children in Grades 3 and 4. A total of 211 children (102 boys) were administered three tasks in narrative and expository genres, respectively, and their written compositions were evaluated in widely used evaluation methods for developing writers: holistic scoring, productivity, and curriculum-based writing scores. Results showed that 54 and 52% of variance in narrative and expository compositions were attributable to true individual differences in writing. Students' scores varied largely by tasks (30.44 and 28.61% of variance), but not by raters. To reach the reliability of .90, multiple tasks and raters were needed, and for the reliability of .80, a single rater and multiple tasks were needed. These findings offer important implications about reliably evaluating children's writing skills, given that writing is typically evaluated by a single task and a single rater in classrooms and even in some state accountability systems. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Reading & Writing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=123224578 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s11145-017-9724-6 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 24 StartPage: 1287 Subjects: – SubjectFull: Composition (Language arts) Type: general – SubjectFull: Dictation (Educational method) Type: general – SubjectFull: Prewriting (Writing process) Type: general – SubjectFull: Research papers (Students) Type: general – SubjectFull: Extinct languages Type: general Titles: – TitleFull: Writing evaluation: rater and task effects on the reliability of writing scores for children in Grades 3 and 4. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Kim, Young-Suk – PersonEntity: Name: NameFull: Schatschneider, Christopher – PersonEntity: Name: NameFull: Wanzek, Jeanne – PersonEntity: Name: NameFull: Gatlin, Brandy – PersonEntity: Name: NameFull: Al Otaiba, Stephanie IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Text: Jun2017 Type: published Y: 2017 Identifiers: – Type: issn-print Value: 09224777 Numbering: – Type: volume Value: 30 – Type: issue Value: 6 Titles: – TitleFull: Reading & Writing Type: main |
| ResultId | 1 |
