View in EDS

Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal?

Saved in:
Bibliographic Details
Title: Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal?
Language: English
Authors: Andrew P. Jaciw (ORCID 0000-0002-9515-7822)
Source: American Journal of Evaluation. 2025 46(1):90-127.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 38
Publication Date: 2025
Document Type: Journal Articles
Reports - Descriptive
Education Level: Elementary Education
Descriptors: Elementary School Students, Elementary School Teachers, Generalization, Test Bias, Test Construction, Quasiexperimental Design, Research Methodology, Randomized Controlled Trials, Causal Models, Comparative Testing, Test Selection, Test Use, Test Validity
Geographic Terms: Tennessee
DOI: 10.1177/10982140241246208
ISSN: 1098-2140
1557-0878
Abstract: By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset by confounded selection into locations. This work shows that this tradeoff can lead to situations where estimates from QEs are less-biased from selection than are estimates from uncompromised XPs when drawing causal generalizations. This work establishes the conditions theoretically, demonstrates the idea empirically, and discusses the implications of the results.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1465063
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1465063
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Andrew+P%2E+Jaciw%22">Andrew P. Jaciw</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-9515-7822">0000-0002-9515-7822</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22American+Journal+of+Evaluation%22"><i>American Journal of Evaluation</i></searchLink>. 2025 46(1):90-127.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 38
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Teachers%22">Elementary School Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Generalization%22">Generalization</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Bias%22">Test Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Construction%22">Test Construction</searchLink><br /><searchLink fieldCode="DE" term="%22Quasiexperimental+Design%22">Quasiexperimental Design</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Methodology%22">Research Methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Randomized+Controlled+Trials%22">Randomized Controlled Trials</searchLink><br /><searchLink fieldCode="DE" term="%22Causal+Models%22">Causal Models</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Testing%22">Comparative Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Selection%22">Test Selection</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Use%22">Test Use</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Tennessee%22">Tennessee</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/10982140241246208
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1098-2140<br />1557-0878
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset by confounded selection into locations. This work shows that this tradeoff can lead to situations where estimates from QEs are less-biased from selection than are estimates from uncompromised XPs when drawing causal generalizations. This work establishes the conditions theoretically, demonstrates the idea empirically, and discusses the implications of the results.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1465063
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1465063
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/10982140241246208
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 38
        StartPage: 90
    Subjects:
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Elementary School Teachers
        Type: general
      – SubjectFull: Generalization
        Type: general
      – SubjectFull: Test Bias
        Type: general
      – SubjectFull: Test Construction
        Type: general
      – SubjectFull: Quasiexperimental Design
        Type: general
      – SubjectFull: Research Methodology
        Type: general
      – SubjectFull: Randomized Controlled Trials
        Type: general
      – SubjectFull: Causal Models
        Type: general
      – SubjectFull: Comparative Testing
        Type: general
      – SubjectFull: Test Selection
        Type: general
      – SubjectFull: Test Use
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Tennessee
        Type: general
    Titles:
      – TitleFull: Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Andrew P. Jaciw
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1098-2140
            – Type: issn-electronic
              Value: 1557-0878
          Numbering:
            – Type: volume
              Value: 46
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: American Journal of Evaluation
              Type: main
ResultId 1