A Protocol to Assess Contextual Factors during Program Impact Evaluation: A Case Study of a STEM Gender Equity Intervention in Higher Education
Saved in:
| Title: | A Protocol to Assess Contextual Factors during Program Impact Evaluation: A Case Study of a STEM Gender Equity Intervention in Higher Education |
|---|---|
| Language: | English |
| Authors: | Suzanne Nobrega (ORCID |
| Source: | American Journal of Evaluation. 2024 45(4):592-609. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 18 |
| Publication Date: | 2024 |
| Sponsoring Agency: | National Institute for Occupational Safety and Health (NIOSH) (DHHS/CDC) |
| Contract Number: | U19OH008857 U19OH012299 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | STEM Education, Gender Differences, Intervention, Program Evaluation, Outcome Measures, Evaluation Methods, Program Effectiveness, Influences, Context Effect, College Faculty, Diversity (Faculty), Work Environment, Case Studies, Teaching Experience, Sex Fairness |
| DOI: | 10.1177/10982140231152281 |
| ISSN: | 1098-2140 1557-0878 |
| Abstract: | Program evaluations that lack experimental design often fail to produce evidence of impact because there is no available control group. Theory-based evaluations can generate evidence of a program's causal effects if evaluators collect evidence along the theorized causal chain and identify possible competing causes. However, few methods are available for assessing competing causes in the program environment. Effect Modifier Assessment (EMA) is a method previously used in smaller-scale studies to assess possible competing causes of observed changes following an intervention. In our case study of a university gender equity intervention, EMA generated useful evidence of competing causes to augment program evaluation. Top-down administrative culture, poor experiences with hiring and promotion, and workload were identified as impeding forces that might have reduced program benefits. The EMA addresses a methodological gap in theory-based evaluation and might be useful in a variety of program settings. |
| Abstractor: | As Provided |
| Entry Date: | 2024 |
| Accession Number: | EJ1452432 |
| Database: | ERIC |
| Abstract: | Program evaluations that lack experimental design often fail to produce evidence of impact because there is no available control group. Theory-based evaluations can generate evidence of a program's causal effects if evaluators collect evidence along the theorized causal chain and identify possible competing causes. However, few methods are available for assessing competing causes in the program environment. Effect Modifier Assessment (EMA) is a method previously used in smaller-scale studies to assess possible competing causes of observed changes following an intervention. In our case study of a university gender equity intervention, EMA generated useful evidence of competing causes to augment program evaluation. Top-down administrative culture, poor experiences with hiring and promotion, and workload were identified as impeding forces that might have reduced program benefits. The EMA addresses a methodological gap in theory-based evaluation and might be useful in a variety of program settings. |
|---|---|
| ISSN: | 1098-2140 1557-0878 |
| DOI: | 10.1177/10982140231152281 |
