View in EDS

The Use of Subsets of Test Questions in an Angoff Standard Setting Method.

Saved in:
Bibliographic Details
Title: The Use of Subsets of Test Questions in an Angoff Standard Setting Method.
Language: English
Authors: Ferdous, Abdullah, Plake, Barbara
Peer Reviewed: N
Page Count: 21
Publication Date: 2003
Document Type: Reports - Research
Speeches/Meeting Papers
Descriptors: Cost Effectiveness, Estimation (Mathematics), Standard Setting (Scoring), Test Items
Abstract: In the Angoff standard setting procedure, subject matter experts (SMEs) estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item comprising the test. In many cases, these item performance estimates are made twice, with information shared with SMEs between estimates. This estimation process can be time-consuming and fatiguing, especially for the SMEs. This study addressed the possibilities of saving time and resources in an Angoff standard setting study. The study used three sets of databases, two from a medical health certification program, with 10 judges for each data et, and one from a financial analyst certification study, with 33 judges. The study shows that using subsets of items, as opposed to the full length test, could be an important consideration that could reduce the time and resources necessary to conduct a standard setting study. The results of this study suggest that 50% of test items may be sufficient to estimate an equivalent minimum passing score in an Angoff setting study. This could result in a substantial saving of time and resources not only for the agency that has been carrying out this activity, but also to the practitioners (panelists) who participate in the standard setting study. (Author/SLD)
Entry Date: 2003
Accession Number: ED475369
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED475369
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED475369
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Use of Subsets of Test Questions in an Angoff Standard Setting Method.
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ferdous%2C+Abdullah%22">Ferdous, Abdullah</searchLink><br /><searchLink fieldCode="AR" term="%22Plake%2C+Barbara%22">Plake, Barbara</searchLink>
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 21
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2003
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research<br />Speeches/Meeting Papers
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Cost+Effectiveness%22">Cost Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+%28Mathematics%29%22">Estimation (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Standard+Setting+%28Scoring%29%22">Standard Setting (Scoring)</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In the Angoff standard setting procedure, subject matter experts (SMEs) estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item comprising the test. In many cases, these item performance estimates are made twice, with information shared with SMEs between estimates. This estimation process can be time-consuming and fatiguing, especially for the SMEs. This study addressed the possibilities of saving time and resources in an Angoff standard setting study. The study used three sets of databases, two from a medical health certification program, with 10 judges for each data et, and one from a financial analyst certification study, with 33 judges. The study shows that using subsets of items, as opposed to the full length test, could be an important consideration that could reduce the time and resources necessary to conduct a standard setting study. The results of this study suggest that 50% of test items may be sufficient to estimate an equivalent minimum passing score in an Angoff setting study. This could result in a substantial saving of time and resources not only for the agency that has been carrying out this activity, but also to the practitioners (panelists) who participate in the standard setting study. (Author/SLD)
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2003
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED475369
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED475369
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 21
    Subjects:
      – SubjectFull: Cost Effectiveness
        Type: general
      – SubjectFull: Estimation (Mathematics)
        Type: general
      – SubjectFull: Standard Setting (Scoring)
        Type: general
      – SubjectFull: Test Items
        Type: general
    Titles:
      – TitleFull: The Use of Subsets of Test Questions in an Angoff Standard Setting Method.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ferdous, Abdullah
      – PersonEntity:
          Name:
            NameFull: Plake, Barbara
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Type: published
              Y: 2003
ResultId 1