View in EDS

The Limits of Inference: Reassessing Causality in International Assessments

Saved in:
Bibliographic Details
Title: The Limits of Inference: Reassessing Causality in International Assessments
Language: English
Authors: David Rutkowski (ORCID 0009-0000-7550-8982), Leslie Rutkowski (ORCID 0000-0003-2868-2202), Greg Thompson (ORCID 0000-0003-0484-8175), Yusuf Canbolat (ORCID 0000-0002-9113-173X)
Source: Large-scale Assessments in Education. 2024 12.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 18
Publication Date: 2024
Document Type: Journal Articles
Reports - Evaluative
Descriptors: International Assessment, Data Use, Causal Models, Educational Research, Validity, Measurement, Research Design
DOI: 10.1186/s40536-024-00197-9
ISSN: 2196-0739
Abstract: This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived from these datasets. The analysis begins with an overview of causality in relation to ILSAs, followed by an examination of randomized control trials and quasi-experimental designs. We juxtapose two quasi-experimental studies demonstrating potential against three studies using ILSA data, revealing significant limitations in causal inference. The discussion addresses the ethical and epistemological challenges in applying quasi-experimental designs to ILSAs, emphasizing the difficulty in achieving robust causal inference. The paper concludes by suggesting a framework for critically evaluating quasi-experimental designs using ILSAs, advocating for a cautious approach in employing these data for causal inferences. We call for a reevaluation of methodologies and conceptual frameworks in comparative education, underscoring the need for a multifaceted approach that combines statistical rigor with an understanding of educational contexts and theoretical foundations.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1419189
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1419189
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: The Limits of Inference: Reassessing Causality in International Assessments
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22David+Rutkowski%22">David Rutkowski</searchLink> (ORCID <externalLink term="http://orcid.org/0009-0000-7550-8982">0009-0000-7550-8982</externalLink>)<br /><searchLink fieldCode="AR" term="%22Leslie+Rutkowski%22">Leslie Rutkowski</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-2868-2202">0000-0003-2868-2202</externalLink>)<br /><searchLink fieldCode="AR" term="%22Greg+Thompson%22">Greg Thompson</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-0484-8175">0000-0003-0484-8175</externalLink>)<br /><searchLink fieldCode="AR" term="%22Yusuf+Canbolat%22">Yusuf Canbolat</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-9113-173X">0000-0002-9113-173X</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Large-scale+Assessments+in+Education%22"><i>Large-scale Assessments in Education</i></searchLink>. 2024 12.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 18
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Evaluative
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22International+Assessment%22">International Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Use%22">Data Use</searchLink><br /><searchLink fieldCode="DE" term="%22Causal+Models%22">Causal Models</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Research%22">Educational Research</searchLink><br /><searchLink fieldCode="DE" term="%22Validity%22">Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement%22">Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Design%22">Research Design</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1186/s40536-024-00197-9
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2196-0739
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived from these datasets. The analysis begins with an overview of causality in relation to ILSAs, followed by an examination of randomized control trials and quasi-experimental designs. We juxtapose two quasi-experimental studies demonstrating potential against three studies using ILSA data, revealing significant limitations in causal inference. The discussion addresses the ethical and epistemological challenges in applying quasi-experimental designs to ILSAs, emphasizing the difficulty in achieving robust causal inference. The paper concludes by suggesting a framework for critically evaluating quasi-experimental designs using ILSAs, advocating for a cautious approach in employing these data for causal inferences. We call for a reevaluation of methodologies and conceptual frameworks in comparative education, underscoring the need for a multifaceted approach that combines statistical rigor with an understanding of educational contexts and theoretical foundations.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1419189
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1419189
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1186/s40536-024-00197-9
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
    Subjects:
      – SubjectFull: International Assessment
        Type: general
      – SubjectFull: Data Use
        Type: general
      – SubjectFull: Causal Models
        Type: general
      – SubjectFull: Educational Research
        Type: general
      – SubjectFull: Validity
        Type: general
      – SubjectFull: Measurement
        Type: general
      – SubjectFull: Research Design
        Type: general
    Titles:
      – TitleFull: The Limits of Inference: Reassessing Causality in International Assessments
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: David Rutkowski
      – PersonEntity:
          Name:
            NameFull: Leslie Rutkowski
      – PersonEntity:
          Name:
            NameFull: Greg Thompson
      – PersonEntity:
          Name:
            NameFull: Yusuf Canbolat
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-electronic
              Value: 2196-0739
          Numbering:
            – Type: volume
              Value: 12
          Titles:
            – TitleFull: Large-scale Assessments in Education
              Type: main
ResultId 1