View in EDS HTML Full Text PDF Full Text

Comparing Single-Level and Multi-Level Analyses with Complex Sampling Weights.

Saved in:
Bibliographic Details
Title: Comparing Single-Level and Multi-Level Analyses with Complex Sampling Weights.
Authors: Shen, Ting1 (AUTHOR) tingshen@mst.edu
Source: Journal of Research on Educational Effectiveness. Apr-Jun2026, Vol. 19 Issue 2, p516-540. 25p.
Subject Terms: *Education statistics, Multilevel models, Statistical weighting, Statistical models, Simulation methods & models
Company/Entity: Trends in International Mathematics & Science Study
Abstract: Large-scale assessment (LSA) data are increasingly utilized to inform education policy and practice worldwide. Because LSA adopts complex sampling designs with unequal probabilities of selection, the use of sampling weights is essential. However, research evidence is scarce, and uncertainty remains regarding how sampling weights should be used, and which approaches are preferrable. This study draws on data from the Early Childhood Longitudinal Study – Kindergarten, the Program for International Student Assessment, and the Trends in International Mathematics and Science Study to compare seven different approaches involving weighting, scaling, and modeling in the context of LSA data. Simulation results indicate that multi-level models without sampling weights have the smallest root mean square error (RMSE) whereas single-level models with overall sampling weights have the largest RMSE. Empirical findings suggest that, overall, weighting, scaling, and modeling choice do not substantially affect statistical significance. Both simulation and empirical analyses reveal that three approaches—multi-level models with school sampling weights, and multi-level models with two-level sampling weights with either size scaling or effective scaling—perform similarly. Discussion and practical recommendations are provided. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Research on Educational Effectiveness is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: ehh
DbLabel: Education Research Complete
An: 193526218
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Comparing Single-Level and Multi-Level Analyses with Complex Sampling Weights.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shen%2C+Ting%22">Shen, Ting</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> tingshen@mst.edu</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Journal+of+Research+on+Educational+Effectiveness%22">Journal of Research on Educational Effectiveness</searchLink>. Apr-Jun2026, Vol. 19 Issue 2, p516-540. 25p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Education+statistics%22">Education statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Multilevel+models%22">Multilevel models</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+weighting%22">Statistical weighting</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+models%22">Statistical models</searchLink><br /><searchLink fieldCode="DE" term="%22Simulation+methods+%26+models%22">Simulation methods & models</searchLink>
– Name: SubjectCompany
  Label: Company/Entity
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Trends+in+International+Mathematics+%26+Science+Study%22">Trends in International Mathematics & Science Study</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Large-scale assessment (LSA) data are increasingly utilized to inform education policy and practice worldwide. Because LSA adopts complex sampling designs with unequal probabilities of selection, the use of sampling weights is essential. However, research evidence is scarce, and uncertainty remains regarding how sampling weights should be used, and which approaches are preferrable. This study draws on data from the Early Childhood Longitudinal Study – Kindergarten, the Program for International Student Assessment, and the Trends in International Mathematics and Science Study to compare seven different approaches involving weighting, scaling, and modeling in the context of LSA data. Simulation results indicate that multi-level models without sampling weights have the smallest root mean square error (RMSE) whereas single-level models with overall sampling weights have the largest RMSE. Empirical findings suggest that, overall, weighting, scaling, and modeling choice do not substantially affect statistical significance. Both simulation and empirical analyses reveal that three approaches—multi-level models with school sampling weights, and multi-level models with two-level sampling weights with either size scaling or effective scaling—perform similarly. Discussion and practical recommendations are provided. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Journal of Research on Educational Effectiveness is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=193526218
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/19345747.2025.2504439
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 25
        StartPage: 516
    Subjects:
      – SubjectFull: Education statistics
        Type: general
      – SubjectFull: Multilevel models
        Type: general
      – SubjectFull: Statistical weighting
        Type: general
      – SubjectFull: Statistical models
        Type: general
      – SubjectFull: Simulation methods & models
        Type: general
      – SubjectFull: Trends in International Mathematics & Science Study
        Type: general
    Titles:
      – TitleFull: Comparing Single-Level and Multi-Level Analyses with Complex Sampling Weights.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shen, Ting
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Text: Apr-Jun2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 19345747
          Numbering:
            – Type: volume
              Value: 19
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Journal of Research on Educational Effectiveness
              Type: main
ResultId 1