View in EDS

Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework

Saved in:
Bibliographic Details
Title: Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework
Language: English
Authors: Kenneth Frank (ORCID 0000-0002-6116-5509), Qinyun Lin, Spiro Maroulis, Shimeng Dai, Contributor, Nicole Jess, Contributor, Hung-Chang Lin, Contributor, Yuqing Liu, Contributor, Sarah Maestrales, Contributor, Ellen Searle, Contributor, Jordan Tait, Contributor
Source: Grantee Submission. 2025.
Peer Reviewed: Y
Page Count: 80
Publication Date: 2025
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305D220022
Document Type: Speeches/Meeting Papers
Reports - Evaluative
Descriptors: Statistical Analysis, Correlation, Predictor Variables, Inferences
Abstract: Sensitivity analyses can inform evidence-based education policy by quantifying the hypothetical conditions necessary to change an inference. Perhaps the most prevalent index used for sensitivity analyses is Oster's (2019) Coefficient of Proportionality (COP). Oster's COP leverages changes in estimated effects and R[superscript 2] when observed covariates are added to a model to quantify how strong selection on "unobserved covariates" would have to be relative to on "observed covariates" to nullify an estimated effect. In this paper, we reconceptualize the COP as a function of unobserved covariates' correlations with the focal predictor (e.g., treatment) and with the outcome. Our correlation-based approach addresses recent critiques of Oster's COP while preserving the comparison of selection on unobserved covariates to selection on observed covariates. As importantly, our expressions do not depend on an analyst's subjective choice of covariates to include in a baseline model, reproduce the exact results from Ordinary Least Squares (OLS) estimates even in finite samples, can be adapted to a threshold for inference based on statistical significance, and can be directly calculated from conventionally reported quantities (e.g., estimated effect, standard error) through the Konfound packages in R or Stata. Thus, for most published studies in the social sciences our COP index can be easily applied and intuitively interpreted.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2025
Accession Number: ED674603
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED674603
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Kenneth+Frank%22">Kenneth Frank</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6116-5509">0000-0002-6116-5509</externalLink>)<br /><searchLink fieldCode="AR" term="%22Qinyun+Lin%22">Qinyun Lin</searchLink><br /><searchLink fieldCode="AR" term="%22Spiro+Maroulis%22">Spiro Maroulis</searchLink><br /><searchLink fieldCode="AR" term="%22Shimeng+Dai%2C+Contributor%22">Shimeng Dai, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Nicole+Jess%2C+Contributor%22">Nicole Jess, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Hung-Chang+Lin%2C+Contributor%22">Hung-Chang Lin, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Yuqing+Liu%2C+Contributor%22">Yuqing Liu, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Sarah+Maestrales%2C+Contributor%22">Sarah Maestrales, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Ellen+Searle%2C+Contributor%22">Ellen Searle, Contributor</searchLink><br /><searchLink fieldCode="AR" term="%22Jordan+Tait%2C+Contributor%22">Jordan Tait, Contributor</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Grantee+Submission%22"><i>Grantee Submission</i></searchLink>. 2025.
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 80
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305D220022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Evaluative
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink><br /><searchLink fieldCode="DE" term="%22Inferences%22">Inferences</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Sensitivity analyses can inform evidence-based education policy by quantifying the hypothetical conditions necessary to change an inference. Perhaps the most prevalent index used for sensitivity analyses is Oster's (2019) Coefficient of Proportionality (COP). Oster's COP leverages changes in estimated effects and R[superscript 2] when observed covariates are added to a model to quantify how strong selection on "unobserved covariates" would have to be relative to on "observed covariates" to nullify an estimated effect. In this paper, we reconceptualize the COP as a function of unobserved covariates' correlations with the focal predictor (e.g., treatment) and with the outcome. Our correlation-based approach addresses recent critiques of Oster's COP while preserving the comparison of selection on unobserved covariates to selection on observed covariates. As importantly, our expressions do not depend on an analyst's subjective choice of covariates to include in a baseline model, reproduce the exact results from Ordinary Least Squares (OLS) estimates even in finite samples, can be adapted to a threshold for inference based on statistical significance, and can be directly calculated from conventionally reported quantities (e.g., estimated effect, standard error) through the Konfound packages in R or Stata. Thus, for most published studies in the social sciences our COP index can be easily applied and intuitively interpreted.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED674603
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED674603
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 80
    Subjects:
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
      – SubjectFull: Inferences
        Type: general
    Titles:
      – TitleFull: Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Kenneth Frank
      – PersonEntity:
          Name:
            NameFull: Qinyun Lin
      – PersonEntity:
          Name:
            NameFull: Spiro Maroulis
      – PersonEntity:
          Name:
            NameFull: Shimeng Dai, Contributor
      – PersonEntity:
          Name:
            NameFull: Nicole Jess, Contributor
      – PersonEntity:
          Name:
            NameFull: Hung-Chang Lin, Contributor
      – PersonEntity:
          Name:
            NameFull: Yuqing Liu, Contributor
      – PersonEntity:
          Name:
            NameFull: Sarah Maestrales, Contributor
      – PersonEntity:
          Name:
            NameFull: Ellen Searle, Contributor
      – PersonEntity:
          Name:
            NameFull: Jordan Tait, Contributor
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 06
              M: 08
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Grantee Submission
              Type: main
ResultId 1