View in EDS

Informing Big Decisions: Analytical Generalizability for Large-Scale Evaluations

Saved in:
Bibliographic Details
Title: Informing Big Decisions: Analytical Generalizability for Large-Scale Evaluations
Language: English
Authors: Estelle Raimondo, Diana Stanescu (ORCID 0009-0004-9985-8716), Santiago Tellez Cañas (ORCID 0000-0001-9536-3709)
Source: Evaluation Review. 2026 50(2):279-311.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 33
Publication Date: 2026
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Generalization, Evaluation Methods, Measurement, Research Problems, Pattern Recognition, Research Design, Institutional Research
DOI: 10.1177/0193841X251380336
ISSN: 0193-841X
1552-3926
Abstract: This article reflects on how the Independent Evaluation Group (IEG) of the World Bank Group grapples with the issue of generalizability in conducting large-scale and complex development evaluations. It discusses a practical framework IEG uses to meet three needs: methodological, institutional, and didactic. IEG evaluations are aimed at informing significant organizational or strategic decisions encompassing broad scopes, assessing extensive portfolios of interventions by the organization across diverse contexts and spanning multiple years. These evaluations are inherently multimethod and need to bridge various logics of generalization. They seek to influence decision-makers, such as boards of directors or senior managerial teams, with the objective of guiding pivotal moments in the organization's trajectory; ensuring accountability for learning, results, or budget expenditures; and synthesizing substantial evidence to distill key success or failure factors for future strategic planning. The defensibility of the methodological scaffolding is paramount to the credibility of the evaluations. The article discusses the challenges inherent in such evaluations, including the need to generate findings that are valid at multiple levels of analysis and the reliance on multitiered mixed-methods approaches. It examines the use of a practical framework to bridge methodological principles and real-world challenges involved in evaluation to inform the design and implementation of large-scale evaluations. The framework is illustrated with examples from IEG's evaluations, and the article explores how practitioners and researchers can apply the framework in other settings to enhance the generalizability of their findings.
Abstractor: As Provided
Entry Date: 2026
Accession Number: EJ1499422
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1499422
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Informing Big Decisions: Analytical Generalizability for Large-Scale Evaluations
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Estelle+Raimondo%22">Estelle Raimondo</searchLink><br /><searchLink fieldCode="AR" term="%22Diana+Stanescu%22">Diana Stanescu</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0004-9985-8716">0009-0004-9985-8716</externalLink>)<br /><searchLink fieldCode="AR" term="%22Santiago+Tellez+Cañas%22">Santiago Tellez Cañas</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9536-3709">0000-0001-9536-3709</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Evaluation+Review%22"><i>Evaluation Review</i></searchLink>. 2026 50(2):279-311.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 33
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Generalization%22">Generalization</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement%22">Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Problems%22">Research Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Pattern+Recognition%22">Pattern Recognition</searchLink><br /><searchLink fieldCode="DE" term="%22Research+Design%22">Research Design</searchLink><br /><searchLink fieldCode="DE" term="%22Institutional+Research%22">Institutional Research</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/0193841X251380336
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0193-841X<br />1552-3926
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This article reflects on how the Independent Evaluation Group (IEG) of the World Bank Group grapples with the issue of generalizability in conducting large-scale and complex development evaluations. It discusses a practical framework IEG uses to meet three needs: methodological, institutional, and didactic. IEG evaluations are aimed at informing significant organizational or strategic decisions encompassing broad scopes, assessing extensive portfolios of interventions by the organization across diverse contexts and spanning multiple years. These evaluations are inherently multimethod and need to bridge various logics of generalization. They seek to influence decision-makers, such as boards of directors or senior managerial teams, with the objective of guiding pivotal moments in the organization's trajectory; ensuring accountability for learning, results, or budget expenditures; and synthesizing substantial evidence to distill key success or failure factors for future strategic planning. The defensibility of the methodological scaffolding is paramount to the credibility of the evaluations. The article discusses the challenges inherent in such evaluations, including the need to generate findings that are valid at multiple levels of analysis and the reliance on multitiered mixed-methods approaches. It examines the use of a practical framework to bridge methodological principles and real-world challenges involved in evaluation to inform the design and implementation of large-scale evaluations. The framework is illustrated with examples from IEG's evaluations, and the article explores how practitioners and researchers can apply the framework in other settings to enhance the generalizability of their findings.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1499422
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1499422
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/0193841X251380336
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 33
        StartPage: 279
    Subjects:
      – SubjectFull: Generalization
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Measurement
        Type: general
      – SubjectFull: Research Problems
        Type: general
      – SubjectFull: Pattern Recognition
        Type: general
      – SubjectFull: Research Design
        Type: general
      – SubjectFull: Institutional Research
        Type: general
    Titles:
      – TitleFull: Informing Big Decisions: Analytical Generalizability for Large-Scale Evaluations
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Estelle Raimondo
      – PersonEntity:
          Name:
            NameFull: Diana Stanescu
      – PersonEntity:
          Name:
            NameFull: Santiago Tellez Cañas
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 0193-841X
            – Type: issn-electronic
              Value: 1552-3926
          Numbering:
            – Type: volume
              Value: 50
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Evaluation Review
              Type: main
ResultId 1