Is England's Office for Students Likely to Falsely Identify Courses as below Threshold on the B3 Progression Metric?
Saved in:
| Title: | Is England's Office for Students Likely to Falsely Identify Courses as below Threshold on the B3 Progression Metric? |
|---|---|
| Language: | English |
| Authors: | Vladislav Areshka (ORCID |
| Source: | Studies in Higher Education. 2025 50(7):1471-1487. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 17 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Foreign Countries, Outcomes of Education, College Graduates, Measurement, Classification, Confidence Testing, Employment Potential, Education Work Relationship, Educational Indicators, Institutional Evaluation, Public Agencies |
| Geographic Terms: | United Kingdom, United Kingdom (England) |
| DOI: | 10.1080/03075079.2024.2382258 |
| ISSN: | 0307-5079 1470-174X |
| Abstract: | Rising inflation and slow growth globally have led to governments focussing on value for money, and in the sphere of higher education policy that is often translated into graduate outcomes. In England, the Office for Students (OfS) has created the progression metric which for full-time, first degree and UK domiciled students requires courses to have 60% of graduates with positive outcomes 15 months after graduation. This paper utilises simulations to evaluate the likelihood that the OfS's approach will fail to identify courses below the threshold (false positives) and incorrectly identify courses at or above the threshold (false negatives). The simulation adapts the levels of positive outcomes within the population (20-90%), sample size (40-1000), percentage sampled (30-90%) and crucially confidence intervals applied (90%, 95% and 99%) to identify courses below threshold. The three main findings are: the choice of 90% CI minimises the likelihood of false positives compared to 95% or 99% CIs; second, larger samples are essential to reduce likelihood of false positives; and third, preferring the 95% confidence interval instead of 99% for taking regulatory action will result in more courses incorrectly identified as below threshold. Governments that choose a statistical approach to regulate educational standards, like the application of confidence intervals to identify provision below quality standards, should utilise simulations to check the likelihood of false positives and false negatives since failure to do so could lead to both: (a) courses below standard not being identified and (b) regulatory action taken incorrectly against provision at or above standards. |
| Abstractor: | As Provided |
| Notes: | https://osf.io/pd43c/?view_only=bb46a701d69846d99c0518616737df4f |
| Entry Date: | 2026 |
| Accession Number: | EJ1502810 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Rising inflation and slow growth globally have led to governments focussing on value for money, and in the sphere of higher education policy that is often translated into graduate outcomes. In England, the Office for Students (OfS) has created the progression metric which for full-time, first degree and UK domiciled students requires courses to have 60% of graduates with positive outcomes 15 months after graduation. This paper utilises simulations to evaluate the likelihood that the OfS's approach will fail to identify courses below the threshold (false positives) and incorrectly identify courses at or above the threshold (false negatives). The simulation adapts the levels of positive outcomes within the population (20-90%), sample size (40-1000), percentage sampled (30-90%) and crucially confidence intervals applied (90%, 95% and 99%) to identify courses below threshold. The three main findings are: the choice of 90% CI minimises the likelihood of false positives compared to 95% or 99% CIs; second, larger samples are essential to reduce likelihood of false positives; and third, preferring the 95% confidence interval instead of 99% for taking regulatory action will result in more courses incorrectly identified as below threshold. Governments that choose a statistical approach to regulate educational standards, like the application of confidence intervals to identify provision below quality standards, should utilise simulations to check the likelihood of false positives and false negatives since failure to do so could lead to both: (a) courses below standard not being identified and (b) regulatory action taken incorrectly against provision at or above standards. |
|---|---|
| ISSN: | 0307-5079 1470-174X |
| DOI: | 10.1080/03075079.2024.2382258 |
