Revisiting the Usage of Alpha in Scale Evaluation: Effects of Scale Length and Sample Size
Saved in:
| Title: | Revisiting the Usage of Alpha in Scale Evaluation: Effects of Scale Length and Sample Size |
|---|---|
| Language: | English |
| Authors: | Leifeng Xiao (ORCID |
| Source: | Educational Measurement: Issues and Practice. 2024 43(2):74-81. |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 8 |
| Publication Date: | 2024 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Measurement, Benchmarking, Item Sampling, Sample Size, Research Methodology, Guidelines, Test Construction |
| DOI: | 10.1111/emip.12604 |
| ISSN: | 0731-1745 1745-3992 |
| Abstract: | Short scales are time-efficient for participants and cost-effective in research. However, researchers often mistakenly expect short scales to have the same reliability as long ones without considering the effect of scale length. We argue that applying a universal benchmark for alpha is problematic as the impact of low-quality items is greater on shorter scales. In this study, we proposed simple guidelines for item reduction using the "alpha-if-item-deleted" procedure in scale construction. An item can be removed if alpha increases or decreases by less than 0.02, especially for short scales. Conversely, an item should be retained if alpha decreases by more than 0.04 upon its removal. For reliability benchmarks, 0.80 is relatively safe in most conditions, but higher benchmarks are recommended for longer scales and smaller sample sizes. Supplementary analyses, including item content, face validity, and content coverage, are critical to ensure scale quality. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Accession Number: | EJ1425083 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Short scales are time-efficient for participants and cost-effective in research. However, researchers often mistakenly expect short scales to have the same reliability as long ones without considering the effect of scale length. We argue that applying a universal benchmark for alpha is problematic as the impact of low-quality items is greater on shorter scales. In this study, we proposed simple guidelines for item reduction using the "alpha-if-item-deleted" procedure in scale construction. An item can be removed if alpha increases or decreases by less than 0.02, especially for short scales. Conversely, an item should be retained if alpha decreases by more than 0.04 upon its removal. For reliability benchmarks, 0.80 is relatively safe in most conditions, but higher benchmarks are recommended for longer scales and smaller sample sizes. Supplementary analyses, including item content, face validity, and content coverage, are critical to ensure scale quality. |
|---|---|
| ISSN: | 0731-1745 1745-3992 |
| DOI: | 10.1111/emip.12604 |
