View in EDS HTML Full Text PDF Full Text

The Critical Role of Anchor Paper Selection in Writing Assessment.

Saved in:
Bibliographic Details
Title: The Critical Role of Anchor Paper Selection in Writing Assessment.
Authors: Osborn Popp, SharonE. (AUTHOR), Ryan, JosephM. (AUTHOR), Thompson, MarilynS. (AUTHOR)
Source: Applied Measurement in Education. 2009, Vol. 22 Issue 3, p255-271. 17p. 4 Charts.
Subjects: Scoring rubrics, Alternative assessment (Education), Test scoring, World class companies, Employment practices, Best practices, Personnel management, Confidence testing
Abstract: Scoring rubrics are routinely used to evaluate the quality of writing samples produced for writing performance assessments, with anchor papers chosen to represent score points defined in the rubric. Although the careful selection of anchor papers is associated with best practices for scoring, little research has been conducted on the role of anchor paper selection in writing assessment. This study examined the consequences of differential selection of anchor papers to represent a common scoring rubric. A set of writing samples was scored under two conditions—one using anchors selected from within grade and one using anchors selected from across three grade levels. Observed ratings were analyzed using three- and four-facet Rasch (one-parameter logistic) models. Ratings differed in magnitude and rank-order, with the difference presumed to be due to the anchor paper conditions and not a difference in overall severity between the rater groups. Results shed light on potential threats to validity within conventional context-dependent scoring practices and raise issues that have not been investigated with respect to the selection of anchor papers, such as the interpretation of results at different grade levels, implications for the assessment of progress over time, and the reliability of anchor paper selection within a scoring context. [ABSTRACT FROM AUTHOR]
Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Psychology and Behavioral Sciences Collection
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: pbh
DbLabel: Psychology and Behavioral Sciences Collection
An: 42746598
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Critical Role of Anchor Paper Selection in Writing Assessment.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Osborn+Popp%2C+SharonE%2E%22">Osborn Popp, SharonE.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Ryan%2C+JosephM%2E%22">Ryan, JosephM.</searchLink> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Thompson%2C+MarilynS%2E%22">Thompson, MarilynS.</searchLink> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Applied+Measurement+in+Education%22">Applied Measurement in Education</searchLink>. 2009, Vol. 22 Issue 3, p255-271. 17p. 4 Charts.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Scoring+rubrics%22">Scoring rubrics</searchLink><br /><searchLink fieldCode="DE" term="%22Alternative+assessment+%28Education%29%22">Alternative assessment (Education)</searchLink><br /><searchLink fieldCode="DE" term="%22Test+scoring%22">Test scoring</searchLink><br /><searchLink fieldCode="DE" term="%22World+class+companies%22">World class companies</searchLink><br /><searchLink fieldCode="DE" term="%22Employment+practices%22">Employment practices</searchLink><br /><searchLink fieldCode="DE" term="%22Best+practices%22">Best practices</searchLink><br /><searchLink fieldCode="DE" term="%22Personnel+management%22">Personnel management</searchLink><br /><searchLink fieldCode="DE" term="%22Confidence+testing%22">Confidence testing</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Scoring rubrics are routinely used to evaluate the quality of writing samples produced for writing performance assessments, with anchor papers chosen to represent score points defined in the rubric. Although the careful selection of anchor papers is associated with best practices for scoring, little research has been conducted on the role of anchor paper selection in writing assessment. This study examined the consequences of differential selection of anchor papers to represent a common scoring rubric. A set of writing samples was scored under two conditions—one using anchors selected from within grade and one using anchors selected from across three grade levels. Observed ratings were analyzed using three- and four-facet Rasch (one-parameter logistic) models. Ratings differed in magnitude and rank-order, with the difference presumed to be due to the anchor paper conditions and not a difference in overall severity between the rater groups. Results shed light on potential threats to validity within conventional context-dependent scoring practices and raise issues that have not been investigated with respect to the selection of anchor papers, such as the interpretation of results at different grade levels, implications for the assessment of progress over time, and the reliability of anchor paper selection within a scoring context. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Applied Measurement in Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=pbh&AN=42746598
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/08957340902984026
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 255
    Subjects:
      – SubjectFull: Scoring rubrics
        Type: general
      – SubjectFull: Alternative assessment (Education)
        Type: general
      – SubjectFull: Test scoring
        Type: general
      – SubjectFull: World class companies
        Type: general
      – SubjectFull: Employment practices
        Type: general
      – SubjectFull: Best practices
        Type: general
      – SubjectFull: Personnel management
        Type: general
      – SubjectFull: Confidence testing
        Type: general
    Titles:
      – TitleFull: The Critical Role of Anchor Paper Selection in Writing Assessment.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Osborn Popp, SharonE.
      – PersonEntity:
          Name:
            NameFull: Ryan, JosephM.
      – PersonEntity:
          Name:
            NameFull: Thompson, MarilynS.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Text: 2009
              Type: published
              Y: 2009
          Identifiers:
            – Type: issn-print
              Value: 08957347
          Numbering:
            – Type: volume
              Value: 22
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Applied Measurement in Education
              Type: main
ResultId 1