The Use of Subsets of Test Questions in an Angoff Standard Setting Method.
Saved in:
| Title: | The Use of Subsets of Test Questions in an Angoff Standard Setting Method. |
|---|---|
| Language: | English |
| Authors: | Ferdous, Abdullah, Plake, Barbara |
| Peer Reviewed: | N |
| Page Count: | 21 |
| Publication Date: | 2003 |
| Document Type: | Reports - Research Speeches/Meeting Papers |
| Descriptors: | Cost Effectiveness, Estimation (Mathematics), Standard Setting (Scoring), Test Items |
| Abstract: | In the Angoff standard setting procedure, subject matter experts (SMEs) estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item comprising the test. In many cases, these item performance estimates are made twice, with information shared with SMEs between estimates. This estimation process can be time-consuming and fatiguing, especially for the SMEs. This study addressed the possibilities of saving time and resources in an Angoff standard setting study. The study used three sets of databases, two from a medical health certification program, with 10 judges for each data et, and one from a financial analyst certification study, with 33 judges. The study shows that using subsets of items, as opposed to the full length test, could be an important consideration that could reduce the time and resources necessary to conduct a standard setting study. The results of this study suggest that 50% of test items may be sufficient to estimate an equivalent minimum passing score in an Angoff setting study. This could result in a substantial saving of time and resources not only for the agency that has been carrying out this activity, but also to the practitioners (panelists) who participate in the standard setting study. (Author/SLD) |
| Entry Date: | 2003 |
| Accession Number: | ED475369 |
| Database: | ERIC |
| Abstract: | In the Angoff standard setting procedure, subject matter experts (SMEs) estimate the probability that a hypothetical randomly selected minimally competent candidate will answer correctly each item comprising the test. In many cases, these item performance estimates are made twice, with information shared with SMEs between estimates. This estimation process can be time-consuming and fatiguing, especially for the SMEs. This study addressed the possibilities of saving time and resources in an Angoff standard setting study. The study used three sets of databases, two from a medical health certification program, with 10 judges for each data et, and one from a financial analyst certification study, with 33 judges. The study shows that using subsets of items, as opposed to the full length test, could be an important consideration that could reduce the time and resources necessary to conduct a standard setting study. The results of this study suggest that 50% of test items may be sufficient to estimate an equivalent minimum passing score in an Angoff setting study. This could result in a substantial saving of time and resources not only for the agency that has been carrying out this activity, but also to the practitioners (panelists) who participate in the standard setting study. (Author/SLD) |
|---|
