General Models for Automated Essay Scoring: Exploring an Alternative to the Status Quo
Saved in:
| Title: | General Models for Automated Essay Scoring: Exploring an Alternative to the Status Quo |
|---|---|
| Language: | English |
| Authors: | Kelly, P. Adam |
| Source: | Journal of Educational Computing Research. 2005 33(1):101-113. |
| Availability: | Baywood Publishing Company, Inc., 26 Austin Avenue, Box 337, Amityville, NY 11701. Tel: 800-638-7819 (Toll Free); Fax: 631-691-1770; e-mail: info@baywood.com. |
| Peer Reviewed: | Y |
| Page Count: | 13 |
| Publication Date: | 2005 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Essays, Models, Writing Evaluation, Validity, Scoring Rubrics, Psychometrics, Comparative Analysis, Writing Skills, Computer Assisted Testing |
| ISSN: | 0735-6331 |
| Abstract: | Powers, Burstein, Chodorow, Fowles, and Kukich (2002) suggested that automated essay scoring (AES) may benefit from the use of "general" scoring models designed to score essays irrespective of the prompt for which an essay was written. They reasoned that such models may enhance score credibility by signifying that an AES system measures the same writing characteristics across all essays. They reported empirical evidence that general scoring models performed nearly as well in agreeing with human readers as did prompt-specific models, the "status quo" for most AES systems. In this study, general and prompt-specific models were again compared, but this time, general models performed as well as or better than prompt-specific models. Moreover, general models measured the same writing characteristics across all essays, while prompt-specific models measured writing characteristics idiosyncratic to the prompt. Further comparison of model performance across two different writing tasks and writing assessment programs bolstered the case for general models. (Contains 4 tables.) |
| Abstractor: | Author |
| Number of References: | 13 |
| Entry Date: | 2006 |
| Access URL: | https://baywood.metapress.com/link.asp?target=contribution&id=19JKUMP512EE4XWE |
| Accession Number: | EJ733955 |
| Database: | ERIC |
| Abstract: | Powers, Burstein, Chodorow, Fowles, and Kukich (2002) suggested that automated essay scoring (AES) may benefit from the use of "general" scoring models designed to score essays irrespective of the prompt for which an essay was written. They reasoned that such models may enhance score credibility by signifying that an AES system measures the same writing characteristics across all essays. They reported empirical evidence that general scoring models performed nearly as well in agreeing with human readers as did prompt-specific models, the "status quo" for most AES systems. In this study, general and prompt-specific models were again compared, but this time, general models performed as well as or better than prompt-specific models. Moreover, general models measured the same writing characteristics across all essays, while prompt-specific models measured writing characteristics idiosyncratic to the prompt. Further comparison of model performance across two different writing tasks and writing assessment programs bolstered the case for general models. (Contains 4 tables.) |
|---|---|
| ISSN: | 0735-6331 |
