View in EDS

Anchor-Based Methods for Judgmentally Estimating Item Difficulty Parameters. LSAC Research Report Series.

Saved in:
Bibliographic Details
Title: Anchor-Based Methods for Judgmentally Estimating Item Difficulty Parameters. LSAC Research Report Series.
Language: English
Authors: Hambleton, Ronald K., Sireci, Stephen G., Swaminathan, H., Xing, Dehui, Rizavi, Saba, Law School Admission Council, Newtown, PA.
Peer Reviewed: N
Page Count: 48
Publication Date: 2003
Document Type: Reports - Research
Descriptors: Adaptive Testing, College Entrance Examinations, Computer Assisted Testing, Difficulty Level, Estimation (Mathematics), Field Tests
Assessment and Survey Identifiers: Law School Admission Test
Abstract: The purposes of this research study were to develop and field test anchor-based judgmental methods for enabling test specialists to estimate item difficulty statistics. The study consisted of three related field tests. In each, researchers worked with six Law School Admission Test (LSAT) test specialists and one or more of the LSAT subtests. The three field tests produced a number of conclusions. A considerable amount was learned about the process of extracting test specialists' estimates of item difficulty. The ratings took considerably longer to obtain than had been expected. Training, initial ratings, and discussion took a considerable amount of time. Test specialists felt they could be trained to estimate item difficulty accurately and, to some extent, they demonstrated this. Average error in the estimates of item difficulty varied from about 11% to 13 %. Also the discussions were popular with the panelists, and almost always resulted in improved item difficulty estimates. By the end of the study, the two expected frameworks that developers thought they might provide test specialists, had merged to one. Test specialists seemed to benefit from the descriptions of items located at three levels of difficulty and from information about the item statistics of many items. Four appendixes describe tasks and contain the field test materials. (Contains 8 tables and 18 references.) (SLD)
Entry Date: 2004
Accession Number: ED482926
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED482926
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED482926
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Anchor-Based Methods for Judgmentally Estimating Item Difficulty Parameters. LSAC Research Report Series.
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hambleton%2C+Ronald+K%2E%22">Hambleton, Ronald K.</searchLink><br /><searchLink fieldCode="AR" term="%22Sireci%2C+Stephen+G%2E%22">Sireci, Stephen G.</searchLink><br /><searchLink fieldCode="AR" term="%22Swaminathan%2C+H%2E%22">Swaminathan, H.</searchLink><br /><searchLink fieldCode="AR" term="%22Xing%2C+Dehui%22">Xing, Dehui</searchLink><br /><searchLink fieldCode="AR" term="%22Rizavi%2C+Saba%22">Rizavi, Saba</searchLink><br /><searchLink fieldCode="AR" term="%22Law+School+Admission+Council%2C+Newtown%2C+PA%2E%22">Law School Admission Council, Newtown, PA.</searchLink>
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 48
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2003
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Adaptive+Testing%22">Adaptive Testing</searchLink><br /><searchLink fieldCode="DE" term="%22College+Entrance+Examinations%22">College Entrance Examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+%28Mathematics%29%22">Estimation (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Field+Tests%22">Field Tests</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22Law+School+Admission+Test%22">Law School Admission Test</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The purposes of this research study were to develop and field test anchor-based judgmental methods for enabling test specialists to estimate item difficulty statistics. The study consisted of three related field tests. In each, researchers worked with six Law School Admission Test (LSAT) test specialists and one or more of the LSAT subtests. The three field tests produced a number of conclusions. A considerable amount was learned about the process of extracting test specialists' estimates of item difficulty. The ratings took considerably longer to obtain than had been expected. Training, initial ratings, and discussion took a considerable amount of time. Test specialists felt they could be trained to estimate item difficulty accurately and, to some extent, they demonstrated this. Average error in the estimates of item difficulty varied from about 11% to 13 %. Also the discussions were popular with the panelists, and almost always resulted in improved item difficulty estimates. By the end of the study, the two expected frameworks that developers thought they might provide test specialists, had merged to one. Test specialists seemed to benefit from the descriptions of items located at three levels of difficulty and from information about the item statistics of many items. Four appendixes describe tasks and contain the field test materials. (Contains 8 tables and 18 references.) (SLD)
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2004
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED482926
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED482926
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 48
    Subjects:
      – SubjectFull: Adaptive Testing
        Type: general
      – SubjectFull: College Entrance Examinations
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Estimation (Mathematics)
        Type: general
      – SubjectFull: Field Tests
        Type: general
      – SubjectFull: Law School Admission Test
        Type: general
    Titles:
      – TitleFull: Anchor-Based Methods for Judgmentally Estimating Item Difficulty Parameters. LSAC Research Report Series.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Law School Admission Council, Newtown, PA.
      – PersonEntity:
          Name:
            NameFull: Hambleton, Ronald K.
      – PersonEntity:
          Name:
            NameFull: Sireci, Stephen G.
      – PersonEntity:
          Name:
            NameFull: Swaminathan, H.
      – PersonEntity:
          Name:
            NameFull: Xing, Dehui
      – PersonEntity:
          Name:
            NameFull: Rizavi, Saba
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2003
ResultId 1