View in EDS HTML Full Text PDF Full Text

Systems-Level Consultation to Improve Intervention Fidelity and Student Outcomes: A Collective Case Study of a Blended Learning Reading Program

Saved in:
Bibliographic Details
Title: Systems-Level Consultation to Improve Intervention Fidelity and Student Outcomes: A Collective Case Study of a Blended Learning Reading Program
Language: English
Authors: Cara M. Bliss (ORCID 0009-0008-8866-0631), Courtenay A. Barrett (ORCID 0000-0001-9258-4725)
Source: Psychology in the Schools. 2025 62(8):2579-2593.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 15
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Descriptors: Blended Learning, Reading Programs, Intervention, Fidelity, Consultation Programs, Program Implementation, Reading Skills, Outcomes of Education, Systems Approach, Elementary School Students
DOI: 10.1002/pits.23490
ISSN: 0033-3085
1520-6807
Abstract: School psychologists have the potential to improve students' reading skills by helping schools select evidence-based programs and then supporting their implementation. Blended learning offers a flexible implementation format to address long-standing challenges implementing and scaling reading programs in schools. The purpose of this study was to explore the unique implementation challenges associated with blended learning programs, how systems-level consultation can be used to support implementation and foster intervention fidelity, and the extent to which intervention fidelity of a blended learning reading program was associated with student reading skills. This collective case study included qualitative and quantitative data from 20 schools, 666 educators, and 6208 students in the U.S. We employed mixed methods, which included inductive thematic analysis of 47 implementation plans and means difference testing to examine differences in educator engagement, student-level intervention fidelity, and student outcomes across districts with varying levels of implementation supports. Results identified several challenges encountered throughout implementation and the types of strategies employed to address those challenges. Further, results suggested that there were greater levels of fidelity and more positive student outcomes in schools accessing systems-level consultation to support implementation.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1477327
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEFLPCvl9lB1k48kJ8maUVzAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHrrf-PBo0PaumAXXAIBEICBmyQP9hhQz5W8GVyW7gAcoXz887tE8uUZM7Dj6ZTtelH6McwRUL24Bz_YN_nZdtoWmaf00IeGVP8I8Aa7c3weCqqlZ3lnnhs1-6RiDpoYEU87CuTeGngRlcZwLZEJ2uvn1AXgSsgPbyeXm6jm1vivA7o-Ikmlk5kDi4lTIBX9stSb1OfnUM3PiHoOgAExAQ2Q1aw7OKsBo27hnqdb
Text:
  Availability: 1
  Value: <anid>AN0186727907;pis01aug.25;2025Jul21.06:39;v2.2.500</anid> <title id="AN0186727907-1">Systems‐Level Consultation to Improve Intervention Fidelity and Student Outcomes: A Collective Case Study of a Blended Learning Reading Program </title> <p>School psychologists have the potential to improve students' reading skills by helping schools select evidence‐based programs and then supporting their implementation. Blended learning offers a flexible implementation format to address long‐standing challenges implementing and scaling reading programs in schools. The purpose of this study was to explore the unique implementation challenges associated with blended learning programs, how systems‐level consultation can be used to support implementation and foster intervention fidelity, and the extent to which intervention fidelity of a blended learning reading program was associated with student reading skills. This collective case study included qualitative and quantitative data from 20 schools, 666 educators, and 6208 students in the U.S. We employed mixed methods, which included inductive thematic analysis of 47 implementation plans and means difference testing to examine differences in educator engagement, student‐level intervention fidelity, and student outcomes across districts with varying levels of implementation supports. Results identified several challenges encountered throughout implementation and the types of strategies employed to address those challenges. Further, results suggested that there were greater levels of fidelity and more positive student outcomes in schools accessing systems‐level consultation to support implementation.</p> <p>Summary: Blended learning programs, which combine face‐to‐face and online instruction, are gaining in popularity.Challenges encountered when implementing a blended learning reading program in schools can be addressed through systems‐level consultation.Systems‐level consultation may improve implementation of a blended learning reading program and positively impact student outcomes.</p> <p>Keywords: blended learning; implementation leadership; implementation supports; intervention fidelity; Lexia Core5; systems‐level consultation</p> <p>Only 33% of fourth‐grade students achieve standards of reading proficiency (National Assessment of Educational Progress NAEP. [<reflink idref="bib57" id="ref1">57</reflink>]) and these students are at heightened risk of experiencing subsequent academic and social difficulties (Connor et al. [<reflink idref="bib13" id="ref2">13</reflink>]). School psychologists have the potential to improve reading outcomes for these students by helping schools select evidence‐based programs (EBPs) and then supporting their implementation (Aarons et al. [<reflink idref="bib3" id="ref3">3</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref4">35</reflink>]). EBPs are standardized procedures, including programs, interventions, and practices, that have been shown to improve students' reading skills through rigorous research studies (e.g., randomized controlled trials; Every Student Succeeds Act [ESSA], 2015). It is imperative to support the development of fundamental reading skills in the early grades to improve long‐term employment, behavioral, and mental health outcomes for students (National Center for Education Statistics [NCES] [<reflink idref="bib44" id="ref5">44</reflink>]), yet how to successfully implement EBPs that support early reading skill development remains a critical challenge facing schools today.</p> <p>A significant body of research documents the challenges of implementing EBPs in schools, which include poor intervention fidelity due to limited time and resources, lack of support, and misalignment between research settings and authentic school contexts (Fixsen et al. [<reflink idref="bib19" id="ref6">19</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref7">35</reflink>]). Research from adjacent fields (e.g., organizational management, public health) illustrates the importance of implementation leadership as a key factor that influences how well EBPs are implemented and sustained (Aarons et al. [<reflink idref="bib1" id="ref8">1</reflink>]). Implementation leadership is defined as behaviors that promote organizational practices or procedures that align with a particular initiative or criterion (in this case, reading EBP implementation) (Aarons et al. [<reflink idref="bib1" id="ref9">1</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref10">35</reflink>]). However, very few, if any, empirical studies have examined how to best facilitate strong implementation leadership to promote the implementation of reading EBPs and subsequent student outcomes, though some research has examined the role of implementation leadership on the implementation of mental health EBPs in schools (Lyon et al. [<reflink idref="bib35" id="ref11">35</reflink>]).</p> <p>Further, no studies to date have examined the process of implementing reading EBPs that employ a blended learning approach, which leverages the strengths of both face‐to‐face and online intervention delivery (Pytash and O'Byrne [<reflink idref="bib54" id="ref12">54</reflink>]). This knowledge gap is critical to address given that approximately 20% of schools in the U.S. implement blended learning programs (Schwartz et al. [<reflink idref="bib56" id="ref13">56</reflink>]) and 35% of classroom teachers employ educational technology to address their students' skill deficits (Digital Promise Global [<reflink idref="bib17" id="ref14">17</reflink>]). Blended learning is gaining in popularity because it offers the promise of individualized, data‐driven instruction in a flexible implementation format that frees up human resources to increase scalability and sustainability (Wilkes et al. [<reflink idref="bib58" id="ref15">58</reflink>]). However, it may be more complex to support the implementation of blended learning given its multiple modalities. The overarching purpose of this collective case study is to understand how implementation supports provided to local leadership teams improved the intervention fidelity[<reflink idref="bib1" id="ref16">1</reflink>] of one blended learning reading EBP, Lexia Core5 Reading, and subsequent student reading outcomes.</p> <hd id="AN0186727907-2">Description of Blended Learning and Lexia Core5 Reading</hd> <p>Blended learning programs combine face‐to‐face instruction with online learning to offer a promising means of addressing poor reading outcomes (Pytash and O'Byrne [<reflink idref="bib54" id="ref17">54</reflink>]). A recent meta‐analysis demonstrated the efficacy of blended learning in improving academic performance (Li and Wang [<reflink idref="bib33" id="ref18">33</reflink>]). The current study examined the implementation of Lexia Core5 Reading (hereafter referred to as Core5), a blended learning program designed to target fundamental literacy skills from Prekindergarten through Grade 5. During the 2022–2023 school year, Core5 was implemented in over 18,300 U.S. schools serving approximately 3.7 million students (Lexia [<reflink idref="bib30" id="ref19">30</reflink>]).</p> <p>Core5 is based on a research‐validated theoretical framework (Gough and Tunmer [<reflink idref="bib22" id="ref20">22</reflink>]; Hoover and Gough [<reflink idref="bib24" id="ref21">24</reflink>]; Perfetti [<reflink idref="bib51" id="ref22">51</reflink>]). Its scope and sequence align with the Essential Elements of Reading identified by the National Reading Panel U.S. & National Institute of Child Health and Human Development U.S. ([<reflink idref="bib45" id="ref23">45</reflink>]) and more recent recommendations for effective elementary reading instruction from the Institute for Educational Sciences (IES; Foorman et al. [<reflink idref="bib20" id="ref24">20</reflink>]). A large body of research evidence demonstrates Core5′s efficacy in improving reading skills among students in Grades PreK‐5 (Lexia [<reflink idref="bib31" id="ref25">31</reflink>]), including several randomized controlled trials (e.g., Hurwitz & Vanacore. [<reflink idref="bib26" id="ref26">26</reflink>]; O'Callaghan et al. [<reflink idref="bib49" id="ref27">49</reflink>]; Schechter et al. [<reflink idref="bib55" id="ref28">55</reflink>]).</p> <p>As a blended learning program, Core5 provides an optimal focus for this collective case study. Blended learning programs combine face‐to‐face instruction with an educator with individualized instruction, assessment, and intervention delivered online. Thus, to implement blended learning programs with fidelity, implementation leaders are tasked with simultaneously supporting the implementation of both face‐to‐face and online educational activities. Despite the unique challenges of balancing these two modalities, blended learning offers a promising solution for the scalability and sustainability challenges of reading EBPs that have plagued education for decades (e.g., Fixsen et al. [<reflink idref="bib19" id="ref29">19</reflink>]). Further, its flexible implementation format can be adapted to meet the needs of a variety of schools with differing structures and schedules (Powell et al. [<reflink idref="bib52" id="ref30">52</reflink>]).</p> <p>Students begin Core5 by completing an adaptive assessment to identify an appropriate starting point in the program based on their current skill level. Then, students work at their own pace through individualized, explicit, systematic online instruction, and complete offline teacher‐directed activities to supplement and extend their learning. The online instruction leverages embedded assessment technology to identify personalized usage targets, which are determined by students' risk level and predicted likelihood of achieving end‐of‐year, grade‐level benchmarks. Personalized usage targets generally range from 20 to 80 min per week depending on students' risk level, with students at greater risk of not meeting end‐of‐year, grade‐level benchmarks receiving higher usage targets. Personalized usage targets are adjusted monthly based on real‐time performance data collected as students progress through the program.</p> <p>When students demonstrate skill mastery by completing a program level, educators can print a certificate of completion to reward student progress. Students can also complete offline activities (Skill Builders) to help generalize their learning. Students that do not make sufficient progress from the online instruction receive scaffolded support in the online program and/or face‐to‐face teacher‐led intervention (Lexia Lessons). Thus, Core5 not only provides evidence‐based assessment and intervention in reading, but also helps schools monitor students' response‐to‐intervention to differentiate instruction and provide increasingly intensive and individualized supports to improve all students' fundamental reading skills (Baron et al. [<reflink idref="bib6" id="ref31">6</reflink>]).</p> <p>As previously described, blended learning programs like Core5 require educators to consider multiple facets of instruction to implement the program with fidelity. A large body of literature demonstrates the importance of intervention fidelity to achieve the effects found in research (Durlak and DuPre [<reflink idref="bib18" id="ref32">18</reflink>]; Prescott et al. [<reflink idref="bib53" id="ref33">53</reflink>]; Schechter et al. [<reflink idref="bib55" id="ref34">55</reflink>]). In the context of Core5 specifically, intervention fidelity is achieved when the following occur: (<reflink idref="bib1" id="ref35">1</reflink>) students meet their personalized usage targets, which requires teachers to allocate sufficient time for those activities, (<reflink idref="bib2" id="ref36">2</reflink>) teachers deliver offline program components (Skill Builders and Lexia Lessons) as appropriate, and (<reflink idref="bib3" id="ref37">3</reflink>) teachers use program data and data‐based instructional recommendations to modify and inform classroom instruction to align with student skill development (Lexia [<reflink idref="bib32" id="ref38">32</reflink>]). Therefore, achieving fidelity requires implementation leaders to support educators across various facets of implementation.</p> <hd id="AN0186727907-3">Implementation Leadership and Systems‐Level Strategies to Improve Fidelity</hd> <p>Research has highlighted the importance of implementation leadership to improve the intervention fidelity of EBPs (Lyon et al. [<reflink idref="bib35" id="ref39">35</reflink>]) and this may be particularly true for complex or multifaceted blended learning programs. Implementation leadership differs from molar or general leadership styles (e.g., transformational leadership; Bass et al. [<reflink idref="bib8" id="ref40">8</reflink>]), which describe the types of behaviors or interactions leaders demonstrate with their employees and have been linked with numerous outcomes, such as employee well‐being (Glisson et al. [<reflink idref="bib21" id="ref41">21</reflink>]; Meidelina et al. [<reflink idref="bib37" id="ref42">37</reflink>]). However, research has suggested that molar leadership styles are too broad to be effectively linked with EBP implementation and that strategic implementation climate, or the perceived importance of the innovation in the organization, is a stronger predictor of intervention fidelity (Aarons et al. [<reflink idref="bib1" id="ref43">1</reflink>]).</p> <p>In schools, implementation leadership consists of seven components: (<reflink idref="bib1" id="ref44">1</reflink>) proactive leadership or the ability to anticipate and address implementation challenges, (<reflink idref="bib2" id="ref45">2</reflink>) knowledgeable leadership or deep understanding of the EBP, (<reflink idref="bib3" id="ref46">3</reflink>) perseverance or the ability to consistently respond to implementation challenges, (<reflink idref="bib4" id="ref47">4</reflink>) support for EBP implementation, (<reflink idref="bib5" id="ref48">5</reflink>) bidirectional communication between leaders and implementers related to the EBP, (<reflink idref="bib6" id="ref49">6</reflink>) vision or how well EBP implementation is integrated into the primary purpose of the school, and (<reflink idref="bib7" id="ref50">7</reflink>) the extent to which the leader is available and responsive to staff needs regarding EBP implementation (Lyon et al. [<reflink idref="bib35" id="ref51">35</reflink>]). Thus, implementation leadership includes behaviors in service of a specific objective (in this case, implementation of a reading EBP) and is critical to foster a climate conducive to EBP implementation (Aarons et al. [<reflink idref="bib1" id="ref52">1</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref53">35</reflink>]).</p> <p>Blended learning is a new and evolving educational approach, and little is known about how to best support its implementation (Mohammed. [<reflink idref="bib41" id="ref54">41</reflink>]). However, a considerable body of research has examined the effectiveness of implementation strategies to increase the intervention fidelity of school‐based EBPs in general. Cook et al. ([<reflink idref="bib14" id="ref55">14</reflink>]) identified 75 unique implementation strategies employed in schools to promote intervention fidelity. These 75 strategies fall under nine conceptual categories: (<reflink idref="bib1" id="ref56">1</reflink>) evaluative and iterative strategies (e.g., progress monitoring), (<reflink idref="bib2" id="ref57">2</reflink>) providing interactive assistance to implementers, (<reflink idref="bib3" id="ref58">3</reflink>) adapting and tailoring the EBP to fit with the local context, (<reflink idref="bib4" id="ref59">4</reflink>) developing relationships with all parties invested in the EBP, (<reflink idref="bib5" id="ref60">5</reflink>) training and educating all parties invested in the EBP, (<reflink idref="bib6" id="ref61">6</reflink>) supporting implementers, (<reflink idref="bib7" id="ref62">7</reflink>) engaging members of the school community, (<reflink idref="bib8" id="ref63">8</reflink>) using financial strategies, and (<reflink idref="bib9" id="ref64">9</reflink>) changing the infrastructure to support implementation. Strong implementation leaders and leadership teams identify the needs of the school and implementers, and then select and employ support strategies based on those needs to improve implementation.</p> <p>Thus, fostering strong implementation leadership is crucial to promoting intervention fidelity and, ultimately, student outcomes. Implementation leaders may be school administrators or other leaders, but may also include other personnel who become "EBP champions" and support implementation efforts within a local context (Lyon et al. [<reflink idref="bib34" id="ref65">34</reflink>]). School psychologists could serve in either of these roles. Importantly, implementation leaders themselves often need support to understand and apply the most effective strategies to facilitate a strong implementation climate, promote fidelity, and address the inevitable challenges of any new school‐based initiative (e.g., Aarons et al. [<reflink idref="bib2" id="ref66">2</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref67">35</reflink>]).</p> <p>Systems‐level consultation delivered to local implementation leadership teams offers one promising method of providing such support. School‐based consultation is frequently used by a variety of professionals, including school psychologists, to assist teachers and other personnel in addressing problems and developing solutions, and research has demonstrated that consultation can improve a variety of student outcomes (Newell et al. [<reflink idref="bib47" id="ref68">47</reflink>]). Consultation is considered an indirect service in that the consultant generally does not provide direct intervention to the student(s) in question (e.g., Kampwirth and Powers [<reflink idref="bib28" id="ref69">28</reflink>]). School psychologists may be particularly well‐positioned to deliver these consultative services, given their preservice training in consultation (e.g., Newman et al. [<reflink idref="bib48" id="ref70">48</reflink>]) and that consultation is outlined as a core competency by the National Association of School Psychologists ([<reflink idref="bib42" id="ref71">42</reflink>]) and the American Psychological Association ([<reflink idref="bib5" id="ref72">5</reflink>]).</p> <p>While traditional models of school‐based consultation have emphasized a triadic relationship between consultant, consultee, and student (Kampwirth and Powers [<reflink idref="bib28" id="ref73">28</reflink>]), a consultative approach can also be applied at the systems level. Systems‐level consultation involves the provision of consultation services to systems‐level stakeholders and is often used to address systemic or organizational barriers and promote implementation of EBPs (Kratochwill et al. [<reflink idref="bib29" id="ref74">29</reflink>]). In systems‐level consultation, the triadic relationship is established between the consultant, one or more systems‐level consultees such as school administrators or teachers, and a systems‐level client; the goal of consultation in this case is to address problems and improve functioning at the systems level so as to benefit individuals (e.g., students) impacted by the system (Meyers et al. [<reflink idref="bib39" id="ref75">39</reflink>]). Systems‐level consultation, and the provision of systems‐level services more broadly, has emerged as a core professional competency for school psychologists, and research attention to systems change efforts in the school psychology literature has grown in recent years (Coleman and Hendricker [<reflink idref="bib12" id="ref76">12</reflink>]). Research has demonstrated that systems‐level consultation can effectively promote the establishment of data systems to monitor fidelity (Barrett [<reflink idref="bib7" id="ref77">7</reflink>]) and promote organizational change (Meyers et al. [<reflink idref="bib38" id="ref78">38</reflink>]). However, no studies to date have examined the types of challenges faced when implementing a blended learning reading EBP and how systems‐level consultation provided to implementation leadership can address those barriers to increase intervention fidelity and student outcomes. Our study fills this gap in the literature.</p> <hd id="AN0186727907-4">Current Study and Research Questions</hd> <p>In the current study, we sought to expand upon prior research by examining the challenges implementation leadership faces when implementing a blended learning reading EBP, the implementation supports used in the context of systems‐level consultation to address those challenges, and the impact of those supports on intervention fidelity and student outcomes. We employ a collective case study methodology to answer the following research questions: (<reflink idref="bib1" id="ref79">1</reflink>) How do implementation supports address the challenges implementation leadership faces when implementing a blended learning reading EBP? (<reflink idref="bib2" id="ref80">2</reflink>) To what extent do schools receiving systems‐level consultation to support implementation have greater levels of intervention fidelity and student reading outcomes compared to schools without implementation supports?</p> <hd id="AN0186727907-5">Methods</hd> <p>We selected collective case study methodology given its examination of multiple cases simultaneously to generate a broad appreciation for the complex implementation process in an everyday setting (Crowe et al. [<reflink idref="bib15" id="ref81">15</reflink>]). Within our case study methodology, we employed mixed methods secondary data analysis, with equal weight placed on the qualitative and quantitative data. We began with qualitative analyses to answer the first research question and identify the types of challenges encountered and the implementation supports used to address those challenges. We then proceeded to quantitative analyses to answer the second research question examining the impact of systems‐level consultation to support implementation (QUAL→QUAN; Palinkas et al. [<reflink idref="bib50" id="ref82">50</reflink>]). Before data analysis, the study was deemed nonhuman subjects research by the Michigan State University Institutional Review Board (IRB). Pseudonyms are used throughout the remainder of the article to maintain the anonymity of the school districts.</p> <hd id="AN0186727907-6">Case Selection and Participants</hd> <p>To increase comparability among the multiple cases selected for inclusion, we bounded this collective case study by geographic region (one Midwestern state), time (the 2022–2023 school year), and scope (inclusive of student outcome data, implementation processes, and systems‐level implementation supports). Case selection was also bounded by access (Crowe et al. [<reflink idref="bib15" id="ref83">15</reflink>]), such that we needed to have access to student outcome data for all schools selected for participation and to systems‐level consultation data for all schools that received this implementation support.</p> <p>Within these bounds, we ultimately selected 3 districts (<emph>N</emph> = 20 schools) for inclusion in this collective case study. One district, hereafter referred to as Washington School District, implemented Core5 in 10 schools (<emph>N</emph> = 3,489 students and 370 educators). Each school in Washington School District received systems‐level consultation intended to support implementation of Core5. One systems‐level consultant provided support to all ten schools in Washington School District. Washington was selected from all possible cases meeting our inclusion criteria because it had the highest number of schools implementing Core5 and the most comprehensive data regarding implementation.</p> <p>We selected two comparison districts from the same state using publicly available demographic information from the 2022–2023 school year obtained from the state department of education's website. We downloaded these data and sorted by district size. We then selected comparison districts based on their similarity to Washington School District in terms of student enrollment, urban location, racial/ethnic composition, the percentage of students qualifying for free or reduced‐price lunch, the percentage of students with disabilities, and the percentage of students who were English Learners. For comparison purposes, we sought to identify schools that had implemented Core5 during the 2022–2023 school year but had not received systems‐level consultation to support implementation. Therefore, for each close match identified, the first author searched Lexia's database to determine whether the district met these criteria. We identified one district that implemented Core5 but did not enroll in systems‐level implementation support (Clinton School District; <emph>N</emph> = 6 schools, 1592 students, 211 educators). A second district (Arlington School District; <emph>N</emph> = 4 schools, 1127 students, 84 educators) implemented Core5 and did enroll in systems‐level implementation support; however, due to a change in district leadership, they did not access (i.e., receive) the available consultation services.</p> <p>In sum, our collective case study included a total sample of 20 schools, 666 educators, and 6208 students across 3 districts who were engaged with Lexia Core5 during the 2022–2023 school year. Table 1 presents additional characteristics of the sample. All three school districts were in their second year of implementation.</p> <p>1 Table Sample and population characteristics.</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Characteristics</th><th>Washington School District (<italic>N</italic> or %)</th><th>Clinton School District (<italic>N</italic> or %)</th><th>Arlington School District (<italic>N</italic> or %)</th></tr></thead><tbody valign="top"><tr><td>School/district characteristics</td><td /><td /><td /></tr><tr><td>Total FTE teachers in district</td><td>515.7</td><td>220.4</td><td>161.9</td></tr><tr><td>Teachers of color</td><td>9.9%</td><td>0.9%</td><td>1.2%</td></tr><tr><td>Total FTE school psychologists in district</td><td>6</td><td>4</td><td>8</td></tr><tr><td>Schools using Core5</td><td>10</td><td>6</td><td>4</td></tr><tr><td>Educators using Core5</td><td>370</td><td>211</td><td>85</td></tr><tr><td>Student characteristics</td><td /><td /><td /></tr><tr><td>Students using Core5</td><td>3,489</td><td>1,592</td><td>1,127</td></tr><tr><td>Race/Ethnicity</td><td /><td /><td /></tr><tr><td>White</td><td>48.7%</td><td>95.8%</td><td>58.6%</td></tr><tr><td>Black/African American</td><td>24.6%</td><td>0.4%</td><td>6.4%</td></tr><tr><td>Hispanic/Latinx</td><td>10.0%</td><td>1.1%</td><td>20.2%</td></tr><tr><td>Multiracial</td><td>16.0%</td><td>2.5%</td><td>14.4%</td></tr><tr><td>Free/reduced meals</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>Students with disabilities</td><td>19.0%</td><td>22.1%</td><td>22.1%</td></tr><tr><td>English learners</td><td>5.6%</td><td>< 0.1%</td><td>6.2%</td></tr><tr><td>Third grade reading proficiency</td><td>37.8%</td><td>85.2%</td><td>46.9%</td></tr></tbody></table> </ephtml> </p> <p>1 <emph>Note:</emph> Pseudonyms are used for the names of the school districts. Reading proficiency reflects performance on the state assessment.</p> <p>2 Abbreviation: FTE, Full‐Time Equivalent.</p> <hd id="AN0186727907-7">Procedure</hd> <p></p> <hd id="AN0186727907-8">Standard Implementation</hd> <p>Students in each school district began using Core5 early in the 2022–2023 school year, with the majority (92%) of the total sample recording their first login by the end of September 2022. Each district used Core5 as a universal (Tier1) supplemental literacy program for students in grades K through 5 in conjunction with the standard English Language Arts curriculum. Students' initial placement (starting) levels within the program were comparable across districts and are summarized in Table 2.</p> <p>2 Table Initial Core5 placement levels by district.</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>District</th><th>At or above grade level (%)</th><th>One level below current grade (%)</th><th>Two or more levels below current grade (%)</th></tr></thead><tbody valign="top"><tr><td>Washington school district</td><td>14%</td><td>33%</td><td>51%</td></tr><tr><td>Clinton school district</td><td>20%</td><td>36%</td><td>43%</td></tr><tr><td>Arlington school district</td><td>9%</td><td>30%</td><td>59%</td></tr></tbody></table> </ephtml> </p> <p>3 <emph>Note:</emph> A small number of students in each district were manually placed into a starting level by their teachers rather than completing the placement assessment. These students are not reflected in the above percentages, which therefore may not sum to exactly 100%.</p> <p>Staff within each district had access to an online progress monitoring dashboard (myLexia), a standard program component available to all Core5 users. The myLexia dashboard allows teachers and administrators to regularly review student‐level progress monitoring data and clearly identify how to differentiate instruction to best meet the needs of all students. The dashboard can be tailored for teachers and administrators so that teachers view their class‐wide data, whereas administrators view school‐ or district‐wide data. Specifically, the myLexia dashboard presents teachers with the following data to monitor student progress: (<reflink idref="bib1" id="ref84">1</reflink>) each students' personalized usage targets and whether they have been met, (<reflink idref="bib2" id="ref85">2</reflink>) which students need offline practice activities to promote skill generalization (Skill Builders), (<reflink idref="bib3" id="ref86">3</reflink>) which students need teacher‐led instruction (Lexia Lessons), and (<reflink idref="bib4" id="ref87">4</reflink>) data‐based recommendations to inform and guide teachers' instructional planning. These data reflect students' real‐time performance in the online program and teachers can use this data to plan for or adapt offline instruction to best meet the needs of each student (Hilliard [<reflink idref="bib23" id="ref88">23</reflink>]; Horn and Staker [<reflink idref="bib25" id="ref89">25</reflink>]). In sum, the myLexia platform is an example of an evaluative and iterative implementation strategy (Cook et al. [<reflink idref="bib14" id="ref90">14</reflink>]) and educator use of the myLexia platform reflects the degree to which this implementation support strategy is delivered (Carroll et al. [<reflink idref="bib11" id="ref91">11</reflink>]). All schools also had access to the Lexia Helpdesk and standard informational materials provided to all Core5 users.</p> <hd id="AN0186727907-9">Systems‐Level Implementation Support</hd> <p>In addition to the standard implementation support described above, schools implementing Lexia programs may opt to receive additional supports provided by systems‐level consultants. These systems‐level consultants are employed by Lexia and have extensive training in literacy and language learning, product knowledge, and implementation best practices, in addition to significant practitioner experience. Consultants leverage this expertise to support school‐level implementation leadership teams. The exact composition of these teams varies by school and is determined on an individual basis based on needs, personnel, and other factors. In general, teams include at least one building‐ and/or district‐level administrator (e.g., principal, curriculum director) in addition to teacher leaders and other relevant personnel (e.g., instructional coaches, reading specialists). As noted above, both Washington and Arlington enrolled in these supports, but only Washington accessed them during the 2022–2023 school year.</p> <p>A systems‐level consultant partnered with implementation leadership teams at each of the 10 schools in the Washington School District that used Core5 during the 2022–2023 school year. One consultant provided services to all schools in our sample. The overarching goal of the systems‐level consultation was to support school teams in improving their implementation leadership skills and selecting and using implementation support strategies identified by Cook et al. ([<reflink idref="bib14" id="ref92">14</reflink>]) to promote intervention fidelity. Specifically, the systems‐level consultant and implementation leadership teams met one to two times before implementation began to develop implementation plans that outlined the unique needs, goals, milestones, and timelines of the school and/or district, as well as the specific resources, training, or other supports necessary for program implementation. Then, throughout the school year, the systems‐level consultant and implementation leadership teams met 2–3 additional times to monitor Core5 intervention fidelity and student progress monitoring data, revise the implementation plan as needed, discuss implementation experiences and challenges, and identify implementation strategies that best met ongoing needs.</p> <p>Each meeting between the systems‐level consultant and implementation leadership teams lasted approximately 1 h. Meetings were led by the systems‐level consultant, who engaged implementation leadership teams in a structured process of goal setting and planning. Throughout the school year, these meetings generally began with a summary overview of Core5 usage and progress monitoring data, and insights gleaned from this data informed subsequent conversations and decisions. Consistent with a collaborative consultation model (Kampwirth and Powers [<reflink idref="bib28" id="ref93">28</reflink>]), the consultant engaged each leadership team as partners in the decision‐making and problem‐solving process.</p> <p>One of the primary implementation strategies employed by these teams was professional learning (PL) to train and educate implementers on how best to implement the multiple components of blended learning programs (Cook et al. [<reflink idref="bib14" id="ref94">14</reflink>]). Together, the systems‐level consultant and local implementation teams collaboratively identified PL needs of implementation leadership and educators and coordinated appropriate PL sessions to address those needs. PL sessions were scheduled in between implementation leadership team meetings and occurred as needed throughout the school year. PL included sessions intended to help leadership and educators engage with the myLexia platform, use student data to plan and/or modify instruction, use school or district data to monitor overall progress, or use the program's offline resources to provide intervention, support, or practice matched to student needs.</p> <hd id="AN0186727907-10">Measures</hd> <p></p> <hd id="AN0186727907-11">Measures of Implementation Support Receipt and Access</hd> <p>Permanent products (e.g., implementation plans, meeting activities and notes) were reviewed and coded to assess systems‐level implementation support receipt, or the extent to which Washington and Arlington School Districts received the systems‐level consultation services as intended. These permanent products were proactively recorded by the systems‐level consultant assigned to provide services to each district. Records indicated that Arlington School District did not access any systems‐level consultation services. Therefore, no implementation support data were available for this district, confirming its appropriateness as a comparison district.</p> <p>For Washington School District, permanent products included implementation support plans developed collaboratively in meetings between the systems‐level consultant and leadership teams at each of the 10 participating schools. These semi‐structured plans included narrative descriptions of goal setting, status updates, professional learning activities, and next step planning. We reviewed and coded implementation plans from Washington School District to assess the quality of the implementation support services provided.</p> <p>A key implementation support available to all schools was the myLexia dashboard, which provides implementers with student‐level progress monitoring data and instructional recommendations. Educator‐level access, or educators' use of the myLexia dashboard, was assessed by (<reflink idref="bib1" id="ref95">1</reflink>) number of logins or the total number of times that an educator logged into the desktop myLexia platform during the 2022–2023 school year and (<reflink idref="bib2" id="ref96">2</reflink>) the number of unique days across the school year in which an educator logged into the desktop myLexia platform. For instance, an educator who logged into the platform three total times, with each login occurring on September 15, would have a value of 3 for number of logins and a value of 1 for days of use.</p> <hd id="AN0186727907-12">Student‐Level Measures of Intervention Fidelity</hd> <p>Student‐level intervention fidelity was assessed by two measures: (<reflink idref="bib1" id="ref97">1</reflink>) the average number of minutes per week that students engaged with the online instruction and (<reflink idref="bib2" id="ref98">2</reflink>) the number of weeks in which students' average usage time was equal to or greater than the personalized usage targets. Personalized usage targets were based on student risk level and generally ranged from 20 to 80 min per week. Students at higher risk of not achieving end‐of‐year, grade‐level benchmarks received higher usage targets. Conversely, students who had already met grade‐level benchmarks had usage targets of 0 min.</p> <hd id="AN0186727907-13">Educator‐Level Measures of Intervention Usage</hd> <p>Educator‐level intervention usage was assessed by the number of offline activities teachers delivered, which included: (<reflink idref="bib1" id="ref99">1</reflink>) offline practice activities to promote generalization (Skill Builders), (<reflink idref="bib2" id="ref100">2</reflink>) teacher‐led intervention activities (Lexia Lessons), and (<reflink idref="bib3" id="ref101">3</reflink>) printing certificates of completion when a student demonstrated mastery in the online instruction. These data were self‐reported by teachers and recorded in the myLexia platform. Educator‐level intervention usage was calculated as the sum of the three types of offline activities.</p> <hd id="AN0186727907-14">Student‐Level Outcome Data</hd> <p>Student‐level outcome data reflected students' growth in reading skills, assessed through two data sources: (<reflink idref="bib1" id="ref102">1</reflink>) number of Core5 program levels completed and (<reflink idref="bib2" id="ref103">2</reflink>) grade‐level material gained. The number of Core5 levels completed was calculated by subtracting the level at which a student started the program from the student's end‐of‐year level. Core5 has 21 total levels that span material from Grades PreK‐5; thus, the maximum number of program levels that could be completed was 21. Grade‐level material gained was calculated by subtracting the grade level of material a student was working on when they started the program from the grade level of material they were working on at the end of the school year. For example, a student who started the year working on material at the 1st grade level and ended the year working on material at the 3rd grade level would have a grade‐level material gained value of 2.</p> <hd id="AN0186727907-15">Data Analysis</hd> <p>At the end of the 2022–2023 school year, the first author downloaded all deidentified secondary data from the Lexia data warehouse. This included the qualitative implementation support data, student‐level intervention fidelity data, educator‐level intervention usage data, and student outcome data. We extracted and organized the qualitative systems‐level implementation support data (in the form of implementation support plans) in an Excel file before data analysis. The quantitative fidelity and outcome data were cleaned (e.g., to remove a small number of educators who transferred from the elementary to high school setting, and a small number of students outside our grade range of interest) and transferred to SPSS for analysis.</p> <p>To ensure the trustworthiness of our qualitative analyses and decrease the potential that our own viewpoints or biases affected the results, we employed several strategies throughout the research process, as outlined by Ahmed ([<reflink idref="bib4" id="ref104">4</reflink>]). First, we established credibility by building rapport between the Lexia staff and external researcher over a prolonged period of time (i.e., approximately 1 year), and explicitly incorporated discussions regarding reflexivity and potential biases throughout the research process. Second, our mixed methods approach triangulated multiple sources of data to support our overarching findings. Third, we sought to provide significant detail regarding our systematic sampling strategy and contextual factors to promote transferability. Finally, we employed a rigorous documentation system, audit trails, and memos to increase the dependability and confirmability of our findings.</p> <hd id="AN0186727907-16">Research Question #1: Qualitative Analysis</hd> <p>To better understand challenges implementation leadership teams face when implementing a blended learning EBP and the implementation supports delivered to address those challenges, we coded the implementation support plans for each of the ten schools in Washington School District using inductive thematic analysis procedures described by Braun and Clarke ([<reflink idref="bib10" id="ref105">10</reflink>]) and Maguire and Delahunt ([<reflink idref="bib36" id="ref106">36</reflink>]). All coding was conducted by the two authors, neither of whom were involved in the systems‐level consultation or implementation process. In total, we coded 47 distinct implementation plans across schools. Each plan documented the discussions and activities that occurred during meetings between the systems‐level consultant and school‐level implementation leadership teams. Each of these meetings lasted approximately 1 h and teams met 2–3 times across the school year. Each plan documented a specific stage of the implementation support process: goal setting (10 out of 47, or 21% of plans coded), status updates (40% of plans coded), and next step planning (38% of plans coded). Additionally, when PL was identified as the appropriate implementation support, implementation plans noted the PL sessions selected.</p> <p>After extracting, cleaning, and organizing all the qualitative data, the first author engaged in repeated reading of the data, noting common patterns or themes (Step 1). After a solid familiarity with the data was established, the first author generated an initial set of codes to reflect themes identified in Step 1. Although we were aware of implementation leadership and implementation strategies before coding, we did not a priori assume that the implementation plans would precisely map onto the constructs described in the literature. Rather, we allowed these codes to emerge from the data. These initial codes were applied to the data in several systematic, comprehensive reviews. Several initial codes were revised, condensed, or deleted based on this iterative review. When the first author was satisfied that the initial coding scheme had been satisfactorily and exhaustively applied to the data, codes were analyzed and sorted into themes (Braun and Clarke [<reflink idref="bib10" id="ref107">10</reflink>]). At this stage, the first author developed a visual representation of the emerging themes to aid in sorting and organizing the information.</p> <p>Next, these themes were reviewed by (<reflink idref="bib1" id="ref108">1</reflink>) rereading all coded segments in relation to identified themes, and (<reflink idref="bib2" id="ref109">2</reflink>) rereading the entire data set in relation to identified themes. As this process progressed, the definitions of individual codes, themes, and their application to the data set were iteratively refined. This process continued until additional reviews did not result in any substantial changes to either theme/code definitions or their application to the data. At this point, the codebook, visual representation of initial thematic structure, and full data set were shared with the second author who served as the second coder. The second author independently analyzed the data set using the codebook developed by the first author.</p> <p>After the second coder had completed her analyses, we compared the results obtained by the first and second coders. Discussion between both coders resulted in the decision to collapse several initial codes to better reflect the thematic structure of the data. All data were recoded using these collapsed codes and agreement between coders was calculated. We aimed to achieve at least 80% agreement (Miles and Huberman [<reflink idref="bib40" id="ref110">40</reflink>]), which we calculated as percent agreement divided by percent agreement plus percent disagreement between coders. Our calculations indicated 81% overall agreement between coders; as we had achieved our predetermined threshold, we proceeded to interpret the results of this analysis.</p> <hd id="AN0186727907-17">Research Question #2: Quantitative Analyses</hd> <p>Our second research question addressed the extent to which there were differences in intervention fidelity and student outcomes across schools with varying levels of systems‐level supports. We calculated district‐level descriptive statistics (<emph>M</emph>, SD) for all educator‐level access and usage and student‐level fidelity and outcome measures for our full sample. We then compared means for each variable across districts using one‐way ANOVAs and/or Kruskal‐Wallis tests as appropriate. When these tests indicated significant differences, we conducted post hoc pairwise comparisons using Fisher's least significant difference (LSD) test (ANOVA) or Dunn's test (Kruskal–Wallis) to determine the nature of these differences. To reduce the risk of Type I error, we adjusted the significance level for all post hoc tests using the Bonferroni correction for multiple comparisons. Because each post hoc test involved 3 comparisons, our adjusted α was 0.05/3 = 0.016.</p> <hd id="AN0186727907-18">Results</hd> <p></p> <hd id="AN0186727907-19">RQ1: Challenges Encountered and Implementation Supports Delivered</hd> <p>Six major themes were identified in the qualitative coding of implementation support plans: Student Usage, Data, Building Knowledge, Blended Learning, Contextual Fit, and Motivation/Buy‐In. These themes, summarized in Table 3, highlight the challenges encountered by implementation leadership teams and the implementation supports delivered to address those challenges.</p> <p>3 Table Qualitative theme definitions and frequency.</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Theme</th><th>Definition</th><th>Frequency</th></tr></thead><tbody valign="top"><tr><td>Student Usage</td><td>Emphasizing student usage and/or progress in Core5 (e.g., helping students meet personalized weekly usage targets, celebrating student usage or progress).</td><td>72%</td></tr><tr><td>Data</td><td>Reviewing and using the data provided through the myLexia platform to monitor student progress, identify needs, and plan or modify instruction.</td><td>43%</td></tr><tr><td>Building Knowledge</td><td>Developing school staff's knowledge and skills related to literacy teaching and learning to facilitate program implementation.</td><td>40%</td></tr><tr><td>Blended Learning</td><td>Implementing the blended learning model, with an emphasis on using offline program components to supplement and extend the instruction provided in the online program.</td><td>28%</td></tr><tr><td>Contextual Fit</td><td>Addressing challenges pertaining to implementing Core5 within each individual school's context.</td><td>26%</td></tr><tr><td>Motivation/Buy‐In</td><td>Recognizing the importance of student motivation to use the program, staff buy‐in or motivation to implement the program, or both.</td><td>19%</td></tr></tbody></table> </ephtml> </p> <p>4 <emph>Note:</emph> Frequency reflects the percent of the 47 distinct implementation plans coded in which both coders agreed that the theme was present.</p> <hd id="AN0186727907-20">Student Usage</hd> <p>Student Usage was the most common theme discussed during meetings between the systems‐level consultant and local implementation leadership teams, and was present in 34 of the 47 implementation plans coded. Teams noted that student usage could be a concern at the student‐ or teacher‐levels: individual students struggled to meet weekly usage targets and some teachers had difficulty allocating sufficient time for students in their class to meet their usage targets.</p> <p>When individual students struggled to meet usage targets, one implementation plan indicated that teachers would meet with these students one‐on‐one to "investigate why and then create a plan to increase progress in relation to their usage." When teachers struggled to allocate time for their students to meet weekly usage targets, implementation supports often involved targeted assistance from a building‐level instructional coach. For example, one implementation plan indicated that a coach would work with these teachers individually to "dig into the 'why' and support them." Leadership teams noted that these individualized student and teacher supports were generally successful in increasing the numbers of students meeting their weekly usage targets.</p> <hd id="AN0186727907-21">Data</hd> <p>Data was the next most common theme and was present in 20 of the 47 implementation plans. Implementation leadership teams discussed the importance of ensuring that educators regularly accessed, interpreted, and used the class‐level data provided in myLexia to monitor student progress, identify needs, and plan or modify instruction. For example, one implementation plan noted, "Teachers are ready to dig deeper into myLexia reports to identify where kids are struggling and plan small‐group instruction that meets those skill needs." School teams also noted the challenges involved in using literacy data to inform instruction. For example, another implementation plan identified a need to "Improve [teachers'] Lexia data analysis skills to design strategic instruction based on the data...We really want [teachers] to use the data in a meaningful way."</p> <p>This challenge was frequently addressed via PL. One implementation plan outlined a PL session in which teachers received guidance and resources to improve their analysis of myLexia reading data and help them identify instructional changes to support students based on this data. Another implementation plan outlined a PL session in which educators received a pedagogical overview of word‐reading automaticity and reading fluency, explored Core5's word‐reading automaticity and reading fluency strand, and were provided links to all Core5 resources related to word‐reading automaticity and reading fluency to support the development of instructional action plans based on student data.</p> <hd id="AN0186727907-22">Building Knowledge</hd> <p>Building Knowledge was the next most common theme and was present in 19 of the 47 implementation plans. Building Knowledge included identifying educator PL needs, scheduling and delivering PL sessions to address those needs, and/or sharing specific resources (e.g., informational or training materials) designed to build the knowledge and skills necessary to implement Core5. Delivering appropriate PL was a common implementation support used to address a variety of challenges, and the Building Knowledge theme highlighted this process. For example, one implementation plan developed at the start of the school year states that the leadership team will "Identify educator needs to help focus professional learning and next steps needed for support." Implementation leadership teams also noted the challenges of translating PL into practice. Another implementation plan addressed this challenge by "Intentionally [planning] time for teachers to process and apply new learning" gained through PL sessions.</p> <p>Leadership teams themselves also needed to build knowledge so that they could best support teachers implementing the program. One implementation plan noted that, in preparation for upcoming grade‐level meetings, building leadership received a reflection guide to "help guide conversations and ask probing questions around best practices" pertaining to Core5. Another plan outlined a process in which building‐level instructional coaches met with grade‐level teams to discuss how they used Core5 to "intentionally plan interventions;" these discussions, in turn, provided the leadership team with a "better idea [of] how to focus PL," for instance by "considering instructional grouping and/or progress monitoring sessions by grade levels."</p> <hd id="AN0186727907-23">Blended Learning</hd> <p>Blended Learning was present in 13 of the implementation plans, highlighting the unique challenges of this pedagogical approach. For example, one implementation plan stated, "Teachers will use Lexia as a teaching tool and not just a computer program.... We'll begin by making sure teachers understand the blended learning model." Another implementation plan noted that the blended learning model required that teachers "are doing more than just putting kids on the computer to meet minutes."</p> <p>Implementation plans outlined strategies intended to ensure that the blended learning model was implemented appropriately. These strategies focused on familiarizing teachers with the program's offline resources (such as Lexia Lessons) and encouraging them to use these resources as needed (e.g., 'Encourage more regular use of Lexia Lessons'). For example, one implementation plan stated, "Lexia Lessons will be delivered as students are flagged on the Needs Instruction tab." Another implementation plan stated, "Teachers should deliver targeted skill instruction with Lexia Lessons when indicated."</p> <hd id="AN0186727907-24">Contextual Fit</hd> <p>The next theme, termed Contextual Fit, was present in 12 of the 47 implementation plans. Challenges pertaining to Contextual Fit included aligning Core5 with other data sources or curriculum products; logistical considerations such as scheduling, rostering, or technological issues; or balancing Core5 with other priorities specific to the school context. For example, one implementation plan illustrated a logistical barrier: "Kids have historically been allowed to take [iPads] home, and we now have many missing. This is a major barrier for implementation of all of our tech programs." Following its identification, this challenge was addressed via a policy change (students in this school were no longer permitted to take iPads home).</p> <p>In another example, an implementation plan highlighted how competing priorities within a school context could impact implementation: "One 3rd grade teacher is piloting [a math program]. Even though she is still using Core5, we expect her usage to decline as she's struggling for time." The need for support strategies addressing challenges related to contextual fit was highlighted in another implementation plan, which stated, "When meeting with district leaders, [the systems‐level consultant] will discuss the need for schools to have guidance around balancing data and instruction between [interim assessment] and Lexia."</p> <hd id="AN0186727907-25">Motivation/Buy‐In</hd> <p>Finally, Motivation/Buy‐In was present in 9 of the implementation plans coded. Some implementation plans highlighted a need to increase motivation or buy‐in. For example, one plan identified a need to "motivate and engage older students" and noted that "this starts with motivating and engaging staff." Other plans celebrated growth in this area as the result of implementation supports. For example, one plan indicated an intention to incorporate school‐wide celebrations, noting that "teachers are excited about Lexia and there's much more buy‐in" and "we want to continue that momentum."</p> <hd id="AN0186727907-26">RQ2: Implementation Supports, Intervention Fidelity, and Student Outcomes</hd> <p>We addressed our second research question by quantitatively examining and comparing educator‐level implementation support access, educator‐level intervention usage, student‐level intervention fidelity, and student‐level outcomes across the 3 districts included in our sample: Washington (which received implementation support via systems‐level consultation services), Arlington (which enrolled in systems‐level consultation services but did not access or receive these services), and Clinton (which did not enroll in or receive systems‐level consultation services).</p> <hd id="AN0186727907-27">Educator‐Level Implementation Support Access</hd> <p>Means and standard deviations for all educator implementation variables are summarized in Table 4. Results from the ANOVA indicated significant differences across districts in both total logins, <emph>F</emph>(<reflink idref="bib2" id="ref111">2</reflink>, 663) = 5.93, <emph>p</emph> = 0.003, <emph>η</emph><sups>2</sups> = 0.018, and days of use, <emph>F</emph>(<reflink idref="bib2" id="ref112">2</reflink>, 663) = 10.55, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.031. Post hoc comparisons using Fisher's LSD indicated that educators in Washington School District had a significantly greater number of logins (<emph>p</emph> = 0.001) and days of use (<emph>p</emph> < 0.001) compared to Clinton School District. Arlington School District also had significantly greater days of use (<emph>p</emph> = 0.002) compared to Clinton School District. There was no significant difference between Washington School District and Arlington School District on either variable (number of logins <emph>p</emph> = 0.88; days of use <emph>p</emph> = 0.92).</p> <p>4 Table Educator implementation, student implementation, and student outcome descriptive statistics.</p> <p> <ephtml> <table><thead valign="bottom"><tr valign="bottom"><th>Characteristics</th><th>Washington school district<ext-link /><sup>1</sup><italic>M</italic> (SD)</th><th>Clinton school district<ext-link /><sup>2</sup><italic>M</italic> (SD)</th><th>Arlington school district<ext-link /><sup>3</sup><italic>M</italic> (SD)</th></tr></thead><tbody valign="top"><tr><td>Educator implementation</td><td /><td /><td /></tr><tr><td>Number of logins</td><td>64.12 (82.25)</td><td>40.51 (80.23)</td><td>65.69 (95.73)</td></tr><tr><td>Days of use</td><td>39.48 (42.30)</td><td>23.49 (40.93)</td><td>40.00 (43.74)</td></tr><tr><td>Offline components delivered</td><td>66.37 (145.33)</td><td>18.75 (43.54)</td><td>49.51 (126.27)</td></tr><tr><td>Student Implementation</td><td /><td /><td /></tr><tr><td>Average minutes per week</td><td>79.55 (37.60)</td><td>49.40 (27.20)</td><td>57.34 (32.76)</td></tr><tr><td>Weeks met usage</td><td>18.52 (10.39)</td><td>14.03 (12.66)</td><td>13.36 (11.77)</td></tr><tr><td>Student Outcomes</td><td /><td /><td /></tr><tr><td>Levels completed</td><td>3.80 (2.60)</td><td>3.08 (2.49)</td><td>2.82 (2.31)</td></tr><tr><td>GLM gained</td><td>1.13 (0.87)</td><td>0.89 (0.80)</td><td>0.85 (0.78)</td></tr></tbody></table> </ephtml> </p> <ulist> <item>5 <emph>Note:</emph> Pseudonyms are used for the names of the school districts.</item> <item>6 Abbreviation: GLM, Grade Level Material.</item> <item>7 1 Washington received systems‐level implementation supports.</item> <item>8 2 Clinton did not enroll in systems‐level supports and did not receive them.</item> <item>9 3 Arlington enrolled in systems‐level supports but did not access or receive them.</item> </ulist> <hd id="AN0186727907-28">Educator‐Level Intervention Usage</hd> <p>Levene's test indicated that the assumption of homogeneity of variance was violated for the offline components delivered variable, <emph>F</emph>(<reflink idref="bib2" id="ref113">2</reflink>, 663) = 25.73, <emph>p</emph> < 0.001. Therefore, we used the Kruskal‐Wallis nonparametric test to examine differences in this variable across districts. Results showed significant differences in offline components delivered, <emph>H</emph>(<reflink idref="bib2" id="ref114">2</reflink>) = 31.98, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.05. Post hoc pairwise comparisons using Dunn's test found that educators in Washington and Arlington School Districts delivered significantly more offline components than educators in Clinton (both <emph>p</emph> < 0.001). The difference between Washington and Arlington was not significant (<emph>p</emph> = 0.94).</p> <hd id="AN0186727907-29">Student‐Level Intervention Fidelity</hd> <p>Means and standard deviations for the student implementation variables are also reported in Table 4. Levene's test indicated that the assumption of homogeneity of variance was violated for both the average minutes per week variable, <emph>F</emph>(<reflink idref="bib2" id="ref115">2</reflink>, 6205) = 73.91, <emph>p</emph> < 0.001, and the weeks met usage variable, <emph>F</emph>(<reflink idref="bib2" id="ref116">2</reflink>, 6205) = 93.04, <emph>p</emph> < 0.001. Therefore, we proceeded with Kruskal‐Wallis tests to examine differences in student‐level intervention fidelity across districts.</p> <p>Results indicated that average minutes per week differed significantly across districts, <emph>H</emph>(<reflink idref="bib2" id="ref117">2</reflink>) = 1000.90, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.16. Post hoc pairwise comparisons using Dunn's test found that students in Washington School District used Core5 for significantly more minutes per week, on average, than students in either Clinton (<emph>p</emph> < 0.001) or Arlington (<emph>p</emph> < 0.001). The difference on average minutes per week also differed significantly between Clinton and Arlington (<emph>p</emph> < 0.001).</p> <p>The number of weeks in which students met recommended usage targets also differed significantly across districts, <emph>H</emph>(<reflink idref="bib2" id="ref118">2</reflink>) = 310.10, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.05. Post hoc comparisons indicated that students in Washington School District met recommended usage targets in significantly more weeks than did students in either Clinton (<emph>p</emph> < 0.001) or Arlington (<emph>p</emph> < 0.001). Clinton and Arlington did not differ significantly on this variable (<emph>p</emph> = 0.26).</p> <hd id="AN0186727907-30">Student‐Level Outcomes</hd> <p>Means and standard deviations for student outcome variables are reported in Table 4. Levene's test indicated that the assumption of homogeneity of variance was violated for the number of levels completed variable, <emph>F</emph>(<reflink idref="bib2" id="ref119">2</reflink>, 6205) = 3.93, <emph>p</emph> = 0.02. Therefore, we examined differences in this variable across districts using the Kruskal‐Wallis test. Results indicated that the number of Core5 levels completed differed significantly across districts, <emph>H</emph>(<reflink idref="bib2" id="ref120">2</reflink>) = 167.84, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.03. Post hoc pairwise comparisons using Dunn's test indicated that students in Washington School District completed significantly more Core5 levels than students in Clinton (<emph>p</emph> < 0.001) or Arlington (<emph>p</emph> < 0.001). The difference in levels completed also differed significantly between Clinton and Arlington (<emph>p</emph> = 0.01).</p> <p>Finally, we examined differences in grade level material (GLM) gained using one‐way ANOVA, as the assumption of homogeneity of variance was satisfied for this variable, <emph>F</emph>(<reflink idref="bib2" id="ref121">2</reflink>, 6205) = 2.70, <emph>p</emph> = 0.07. Results indicated significant differences in GLM gained across districts, <emph>F</emph>(<reflink idref="bib2" id="ref122">2</reflink>,6205) = 70.67, <emph>p</emph> < 0.001, <emph>η</emph><sups>2</sups> = 0.02. Post hoc LSD tests indicated that GLM gains in Washington School District were significantly higher than in Clinton (<emph>p</emph> < 0.001) or Arlington (<emph>p</emph> < 0.001). The difference in GLM gained between Clinton and Arlington was not significant (<emph>p</emph> = 0.21).</p> <hd id="AN0186727907-31">Discussion</hd> <p>Adopting and supporting EBP implementation is essential to improve students' reading proficiency. Despite the critical need to improve student reading skills, research has highlighted the challenges of adopting, implementing, and scaling evidence‐based literacy programs (e.g., Fixsen et al. [<reflink idref="bib19" id="ref123">19</reflink>]). Blended learning programs offer a unique approach to tailor supports for individual students in a flexible, scalable format. This study was the first to explore how systems‐level consultation services can support implementation leadership, increase intervention fidelity of a blended learning program, and ultimately improve student reading skills. We employed mixed methods to identify the challenges encountered during implementation, understand how implementation supports addressed those challenges, and the extent to which these supports were associated with increased intervention fidelity and student outcomes. Our results align with prior research regarding effective implementation leadership and implementation supports (Lyon et al. [<reflink idref="bib34" id="ref124">34</reflink>]; Cook et al. [<reflink idref="bib14" id="ref125">14</reflink>]) and extends this literature by highlighting the unique dynamic of blended learning reading programs, which we describe next.</p> <hd id="AN0186727907-32">Challenges Implementing Blended Learning Reading Programs and How to Address Them</hd> <p>Our qualitative results aligned with prior research, suggesting that educator knowledge, skills, and buy‐in are common challenges faced during school‐based EBP implementation (Fixsen et al. [<reflink idref="bib19" id="ref126">19</reflink>]), that implementation strategies are needed to address those challenges (Cook et al. [<reflink idref="bib14" id="ref127">14</reflink>]), and that implementation leadership is needed to ensure that those strategies are delivered to the educators who need them (Lyon et al. [<reflink idref="bib35" id="ref128">35</reflink>]). Our results identified topic areas that implementation leadership may want to be aware of when implementing blended learning reading programs, as these may represent common areas in need of additional training and coaching for educators.</p> <p>First, the most common challenge faced by implementation leadership pertained to ensuring that students had adequate time to engage with the EBP. This challenge was evident at both the student and teacher levels, and is consistent with a large body of research identifying time as an important resource that can impact implementation (e.g., Fixsen et al. [<reflink idref="bib19" id="ref129">19</reflink>]). Here, this challenge was identified in the context of regular reviews of progress monitoring data (inclusive of usage time) that were coordinated by the systems‐level consultant. When usage concerns surfaced, implementation leadership identified the level at which the barrier occurred (e.g., student or teacher) and addressed these barriers via individualized supports. When students were struggling to meet usage targets despite being provided adequate time to do so, teachers were encouraged to meet with students individually to trouble‐shoot and offer support. When teachers struggled to provide students adequate time, they were supported by building‐level instructional coaches who met with teachers individually to understand the time barriers they were facing and develop a plan to address these barriers. Thus, systems‐level consultants supported implementation leadership in using evaluative and iterative strategies and interactive assistance to implementers (Cook et al. [<reflink idref="bib14" id="ref130">14</reflink>]) to identify and address this important barrier.</p> <p>Second, several themes highlighted the need for knowledge of the specific content of the EBP, in this case, knowledge about literacy and blended learning. More specifically, these themes highlighted how complex it is to foster student reading skills (Build Knowledge), use data to inform and adjust instruction (Data), balance and integrate individualized online activities with in‐person whole group instruction (Blended Learning), and manage implementation within the context of multiple competing demands and priorities (Contextual Fit). Implementation leaders must have sufficient knowledge of these topics to answer questions about EBP implementation (Lyon et al. [<reflink idref="bib35" id="ref131">35</reflink>]) and qualitative findings highlighted how systems‐level consultants could simultaneously identify needs and deliver PL to leaders and educators managing these activities in the classroom (Cook et al. [<reflink idref="bib14" id="ref132">14</reflink>]), as well as foster knowledgeable leadership during systems‐level consultation sessions (Lyon et al. [<reflink idref="bib35" id="ref133">35</reflink>]).</p> <p>Next, results highlighted the need for buy‐in across leadership, teachers, and students with implementation plans identifying specific challenges with buy‐in among teachers and students, and how to address those challenges. Cook et al. ([<reflink idref="bib14" id="ref134">14</reflink>]) highlight the need to engage all parties affected by EBP implementation and implementation plans often noted the need to engage teachers and students individually through instructional coaches and other leaders. This was further nuanced by the buy‐in needed to implement blended learning with fidelity, not just to place students on technology for a predetermined number of minutes per week. Instead, it was critical for educators to shift their view of blended learning as a means to better understand their students' instructional needs, address those needs by adjusting their classroom instruction, and provide more intensive, face‐to‐face instruction to students that needed it.</p> <p>In sum, these themes or challenges were typically addressed through a cadre of strategies identified by Cook et al. ([<reflink idref="bib14" id="ref135">14</reflink>]) including both in‐person and online professional learning, engaging and developing relationships with all parties, supporting implementers through praise and other incentive structures, tailoring the EBP to fit the local context, and providing interactive assistance to implementers through a combination of activities such as consultation and coaching.</p> <hd id="AN0186727907-33">Systems‐Level Implementation Supports Are Associated With Improved Outcomes</hd> <p>Results from our quantitative analyses provide promising evidence that systems‐level consultation can foster implementation leadership and the use of implementation strategies (Cook et al. [<reflink idref="bib14" id="ref136">14</reflink>]; Lyon et al. [<reflink idref="bib35" id="ref137">35</reflink>]), improve student‐level intervention fidelity and educator‐level intervention usage, and positively impact student reading outcomes. Students in schools that received implementation support via systems‐level consultation demonstrated significantly higher levels of intervention fidelity than students in either comparison district. They used the program for more minutes per week, on average, and met personalized usage targets in significantly more weeks than students in districts that did not receive implementation support. Further, students in schools that received implementation support also made substantially more progress in the program than students in comparison schools. Specifically, students in Washington School District advanced almost 4 of the program's 21 total levels, on average, across the school year compared to students in comparison districts who completed, on average, 3 or fewer levels. Additionally, students in Washington School District gained more than one grade level across the school year, while students in comparison districts gained less than one grade level in the same year. These results are significant because this study is the first to provide preliminary evidence that systems‐level consultation to support implementation of a blended learning reading program is empirically linked with positive student outcomes.</p> <p>The relationship between systems‐level services and educator‐level engagement with the program is somewhat less clear. Washington School District, who received implementation support, had high levels of educator access and usage on all measures, including access of the myLexia platform and delivery of offline reading lessons. However, Arlington School District also had high levels of myLexia platform use, but this did not translate into greater delivery of offline reading lessons, student‐level fidelity or program usage, or student reading outcomes. It may be that educators need to do more than just access progress monitoring data on an online platform; rather they need PL and coaching to translate those progress monitoring data into instructional strategies that can be implemented in the classroom (Joyce and Showers [<reflink idref="bib27" id="ref138">27</reflink>]).</p> <p>This hypothesis aligns with our qualitative results, which illustrated that incorporating literacy data from a blended learning program into classroom instruction was a common challenge faced by educators. When these needs were identified by implementation leadership during systems‐level consultation, leaders were able to engage the consultants to provide PL to their educators to improve these skills and increase fidelity. Importantly, PL sessions were selected based on identified needs of implementers, provided coaching or expert support to address those needs, and included adequate time for participants to reflect on what they learned and plan for applications to classroom practice (e.g., Darling‐Hammond et al. [<reflink idref="bib16" id="ref139">16</reflink>]).</p> <p>Consistent with the notion of Contextual Fit, contextual factors may also have impacted the fidelity and student outcomes in Arlington School District. Despite the availability of systems‐level implementation supports for schools in this district, records indicated that Arlington schools did not access or receive these supports. It is possible that this lack of systems‐level consultation receipt was due to a change in district leadership that occurred just before the school year in which this study was conducted. In this case, the prior administrator enrolled in the systems‐level supports but the subsequent administrator did not have buy‐in regarding Core5 implementation or systems‐level services. In their second year of implementation, educators in Arlington may have been motivated to individually log‐in and review their progress monitoring data, but without implementation leadership and PL delivered by systems‐level consultants, this willingness did not translate to intervention usage or student outcomes.</p> <hd id="AN0186727907-34">Limitations and Suggestions for Future Research</hd> <p>Despite the merits of this study, there were also several limitations which are important to note. First, although data from 20 schools, 666 educators, and 6,208 students were included in this study, this is a very limited representation of the 18,300 schools and approximately 3.7 million students that are currently engaged with Core5 in the U.S. This small sample size was necessary to employ a mixed methods approach, which included a resource‐intensive process to individually code 47 implementation plans. However, the small sample size from one state minimizes the generalizability of the study. This will be important to address in future research given the growing popularity of blended learning (e.g., Wilkes et al. [<reflink idref="bib58" id="ref140">58</reflink>]).</p> <p>Second, the study lacked experimental design, such that we cannot make causal inferences that the systems‐level supports were the primary mechanism for improved intervention fidelity and student outcomes. It is possible that any number of other factors may have contributed to the improved outcomes, including selection bias or other district‐level reading initiatives. Indeed, it is worth noting that the systems‐level consultation services described here require an additional financial commitment on the part of participating districts and thus may indicate increased investment in the success of program implementation. Future research that controls for these and other potential confounds via random assignment to condition will be imperative in establishing the true impact of these services.</p> <p>Additionally, our reliance on indirect measures to assess educator engagement with the online platform and delivery of offline program components is a limitation of this study, particularly the use of self‐report data to assess educator intervention usage. Limitations of self‐report data include the potential for social desirability bias, reporter error, and the overall subjectivity of data obtained in this manner (e.g., Nederhof [<reflink idref="bib46" id="ref141">46</reflink>]; Bauhoff [<reflink idref="bib9" id="ref142">9</reflink>]). Future research should address this limitation by incorporating direct observation methods to assess educator implementation fidelity of offline program components delivered to students.</p> <p>Further, we did not employ explicit measures of implementation leadership (Lyon et al. [<reflink idref="bib35" id="ref143">35</reflink>]) nor school‐based implementation strategies (Cook et al. [<reflink idref="bib14" id="ref144">14</reflink>]). However, we believe our approach, which used secondary data, was a more authentic assessment of the implementation process of blended learning reading programs in contemporary schools, thereby increasing external validity. Future research may wish to employ a greater variety of qualitative methods, such as interviews, focus groups, and direct observations, to gain deeper understanding of how implementation leadership is developed, the unique challenges implementing blended learning reading programs across diverse contexts, and how blended learning reading programs translate into student outcomes. Future research may also wish to employ a greater variety of quantitative data, including standardized assessments of academic skills external to the EBP (e.g., state assessment data, computer adaptive programming), validated measures of implementation leadership (Lyon et al. [<reflink idref="bib35" id="ref145">35</reflink>]), and other validated measures of leadership and organizational factors to better understand the systems‐level implementation process of blended learning reading programs. Finally, future research may also wish to employ more rigorous causal research designs and quantitative methods to increase our confidence that systems‐level consultation to support implementation does, in fact, improve educator engagement and student‐level intervention fidelity, in turn improving student reading skills.</p> <hd id="AN0186727907-35">Conclusion</hd> <p>School psychologists must support students' reading skill development, and they may do this at‐scale by supporting schools to adopt and implement blended learning reading programs. This study was the first to document the unique challenges faced by implementation leadership throughout the implementation process, explore how systems‐level consultation and implementation strategies can be used to improve intervention fidelity, and the extent to which intervention fidelity is associated with improved student outcomes. Given school psychologists' competency in systems change and systems‐level consultation (National Association of School Psychologists. [<reflink idref="bib43" id="ref146">43</reflink>]), they are well positioned to deliver services such as those described here. Results of this study highlighted the importance of implementation leadership to enact strategies to support educators and students. This study offers a starting point for researchers and practitioners to deepen their understanding of the unique role of implementation leadership in fostering student reading proficiency.</p> <hd id="AN0186727907-36">Acknowledgments</hd> <p>The authors wish to acknowledge Jennie Tober, John Abner, Raj Chattergoon, and Liz Brooke for their assistance in conducting this study.</p> <hd id="AN0186727907-37">Ethics Statement</hd> <p>This study was conducted in accordance with the ethical standards described in the 1964 Declaration of Helsinki and determined to be exempt by the Michigan State University Human Research Protection Program, Office of Regulatory Affairs on August 21, 2023.</p> <hd id="AN0186727907-38">Consent</hd> <p>When schools purchase Lexia® products or services, they provide consent for their anonymized data to be used by Lexia for research purposes.</p> <hd id="AN0186727907-39">Conflicts of Interest</hd> <p>This submission evaluates the effectiveness of a commercial product and services. Cara M. Bliss is employed by Lexia®, the company offering the product and services evaluated herein. Cara M. Bliss does not receive commission on sales of these products or services. The results and conclusions presented in this article are those of the authors and do not necessarily reflect the views of Lexia®. The other author declares no conflicts of interest.</p> <hd id="AN0186727907-40">Data Availability Statement</hd> <p>Supporting data for this study are not available. This study was conducted using data Lexia receives as part of its educational programming, through which Lexia receives consent from the data subjects to use their anonymized data solely for Lexia's own research and analysis purposes. Participants have not given specific consent to share such data, in any form, with third parties.</p> <ref id="AN0186727907-41"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref8" type="bt">1</bibl> <bibtext> We use the term intervention fidelity here to describe the degree to which an EBP (Lexia Core5 Reading) was delivered as intended. We chose this term to differentiate between the fidelity of an intervention and the fidelity of an implementation support strategy, which is consistent with the broader implementation science literature. However, it is important to note that Core5 is designed to support students of all abilities in Grades PreK‐5 (universal prevention) regardless of whether a student has been identified as requiring formal intervention.</bibtext> </blist> </ref> <ref id="AN0186727907-42"> <title> References </title> <blist> <bibtext> Aarons, G. A., M. G. Ehrhart, and L. R. Farahnak. 2014. " The Implementation Leadership Scale (ILS): Development of a Brief Measure of Unit Level Implementation Leadership." Implementation Science 9, no. 45 : 1. https://doi.org/10.1186/1748-5908-9-45.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref36" type="bt">2</bibl> <bibtext> Aarons, G. A., M. G. Erhhart, L. R. Farahnak, and M. Hurlburt. 2015. " Leadership and Organizational Change for Implementation (LOCI): A Randomized Mixed Method Pilot Study of a Leadership and Organization Development Intervention for Evidence‐Based Practice Implementation." Implementation Science 10, no. 11 : 1 – 12. https://doi.org/10.1186/s13012-014-0192-y.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref3" type="bt">3</bibl> <bibtext> Aarons, G. A., M. Hurlburt, and S. M. Horwitz. 2011. " Advancing a Conceptual Model of Evidence‐Based Practice Implementation in Public Service Sectors." Administration and Policy in Mental Health and Mental Health Services Research 38, no. 1 : 4 – 23.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref47" type="bt">4</bibl> <bibtext> Ahmed, S. K. 2024. " The Pillars of Trustworthiness in Qualitative Research." Journal of Medicine, Surgery, and Public Health 2 : 100051. https://doi.org/10.1016/j.glmedi.2024.100051.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref48" type="bt">5</bibl> <bibtext> American Psychological Association. 2006. Guidelines and Principles for Accreditation of Programs in Professional Psychology (G&P). <ulink href="http://www.apa.org/ed/accreditation/about/policies/guiding-principles.pdf">http://www.apa.org/ed/accreditation/about/policies/guiding-principles.pdf</ulink>.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref31" type="bt">6</bibl> <bibtext> Baron, L. S., T. P. Hogan, R. L. Schechter, P. E. Hook, and E. C. Brooke. 2019. " Can Educational Technology Effectively Differentiate Instruction for Reader Profiles? " Reading and Writing 32 : 2327 – 2352. https://doi.org/10.1007/s11145-019-09949-4.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref50" type="bt">7</bibl> <bibtext> Barrett, C. A. 2021. " Using Systems‐Level Consultation to Establish Data Systems to Monitor Coaching in Schools: A Framework for Practice." Journal of Educational and Psychological Consultation 31 : 411 – 437. https://doi.org/10.1080/10474412.2020.1830409.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref40" type="bt">8</bibl> <bibtext> Bass, B. M., and B. J. Avolio. 1993. " Transformational Leadership and Organizational Culture." PAQ 1993 17, no. 1 : 112 – 121, <ulink href="http://www.jstor.org/stable/40862.298">http://www.jstor.org/stable/40862.298</ulink>.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref64" type="bt">9</bibl> <bibtext> Bauhoff, S. 2023. " Self‐Report Bias in Estimating Cross‐Sectional and Treatment Effects." In Encyclopedia of Quality of Life and Well‐Being Research edited by F. Maggino, 6277 – 6279. Springer. https://doi.org/10.1007/978-3-031-17299-1_4046.</bibtext> </blist> <blist> <bibtext> Braun, V., and V. Clarke. 2006. " Using Thematic Analysis in Psychology." Qualitative Research in Psychology 3, no. 2 : 77 – 101. https://doi.org/10.1191/1478088706qp063oa.</bibtext> </blist> <blist> <bibtext> Carroll, C., M. Patterson, S. Wood, A. Booth, J. Rick, and S. Balain. 2007. " A Conceptual Framework for Implementation Fidelity." Implementation Science 2 : 40. https://doi.org/10.1186/1748-5908-2-40.</bibtext> </blist> <blist> <bibtext> Coleman, S. L., and E. Hendricker. 2020. " An Examination of Systems Change Research in School Psychology." Contemporary School Psychology 24, no. 4 : 488 – 507. https://doi.org/10.1007/s40688-020-00295-9.</bibtext> </blist> <blist> <bibtext> Connor, C. M., P. A. Alberto, D. L. Compton, and R. E. O'Connor. 2014. Improving Reading Outcomes for Students with or At Risk for Reading Disabilities: A Synthesis of the Contributions from the Institute of Education Sciences Research Centers (NCSER 2014‐3000). National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education.</bibtext> </blist> <blist> <bibtext> Cook, C. R., A. R. Lyon, J. Locke, T. Waltz, and B. J. Powell. 2019. " Adapting a Compilation of Implementation Strategies to Advance School‐Based Implementation Research." Prevention Science 20 : 914 – 935. https://doi.org/10.1007/s11121-019-01017-1.</bibtext> </blist> <blist> <bibtext> Crowe, S., K. Cresswell, A. Robertson, G. Huby, A. Avery, and A. Sheikh. 2011. " The Case Study Approach." BMC Medical Research Methodology 11 : 100. https://doi.org/10.1186/1471-2288-11-100.</bibtext> </blist> <blist> <bibtext> Darling‐Hammond, L., M. E. Hyler, and M. Gardner. 2017. Effective Teacher Professional Development. Learning Policy Institute.</bibtext> </blist> <blist> <bibtext> Digital Promise Global 2019. Learning in the 21 st Century: How the American Public, Parents, and Teachers View Students' Potential and Their Learning Experience. Digital Promise Global.</bibtext> </blist> <blist> <bibtext> Durlak, J. A., and E. P. DuPre. 2008. " Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation." American Journal of Community Psychology 41, no. 3–4 : 327 – 350. https://doi.org/10.1007/s10464-008-9165-0.</bibtext> </blist> <blist> <bibtext> Fixsen, D. L., S. F. Naoom, K. A. Blasé, R. M. Friedman, and F. Wallace. 2005. Implementation Research: A Synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).</bibtext> </blist> <blist> <bibtext> Foorman, B., N. Beyler, and K. Borradaile, et al. 2016. " National Center for Education Evaluation and Regional Assistance (NCEE)." In Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade. Institute of Education Sciences : U.S. Department of Education.</bibtext> </blist> <blist> <bibtext> Glisson, C., J. Landsverk, S. Schoenwald, et al. 2008. " Assessing the Organizational Social Context (OSC) of Mental Health Services: Implications for Research and Practice." Administration and Policy in Mental Health and Mental Health Services Research 35, no. 1–2 : 98 – 113.</bibtext> </blist> <blist> <bibtext> Gough, P. B., and W. E. Tunmer. 1986. " Decoding, Reading, and Reading Disability." Remedial and Special Education 7 : 6 – 10.</bibtext> </blist> <blist> <bibtext> Hilliard, A. T. 2015. " Global Blended Learning Practices for Teaching and Learning, Leadership and Professional Development." Journal of International Education Research (JIER) 11 : 179 – 188.</bibtext> </blist> <blist> <bibtext> Hoover, W. A., and P. B. Gough. 1990. " The Simple View of Reading." Reading and Writing 2 : 127 – 160.</bibtext> </blist> <blist> <bibtext> Horn, M. B., and H. Staker. 2011. The Rise of K–12 Blended Learning. Innosight Institute.</bibtext> </blist> <blist> <bibtext> Hurwitz, L. B., and K. P. Vanacore. 2023. " Educational Technology In Support of Elementary Students With Reading or Language‐Based Disabilities: A Cluster Randomized Control Trial." Journal of Learning Disabilities 56 : 453 – 466. https://doi.org/10.1177/00222194221141093.</bibtext> </blist> <blist> <bibtext> Joyce, B. R., and B. Showers. 2002. Student Achievement Through Staff Development, Vol. 3. Association for Supervision and Curriculum Development.</bibtext> </blist> <blist> <bibtext> Kampwirth, T. J., and K. M. Powers. 2015. Collaborative Consultation in the Schools: Effective Practices for Students With Learning and Behavior Problems. Pearson.</bibtext> </blist> <blist> <bibtext> Kratochwill, T. R., M. R. Altschaefl, and B. Bice‐Urbach. 2008. " Best practices in School‐Based Problem‐Solving Consultation: Applications in Prevention and Intervention Systems." In Best Practices in School Psychology, 1673 – 1688. National Association of School Psychologists.</bibtext> </blist> <blist> <bibtext> Lexia 2023a. Lexia Core5 Reading US National Progress Report with Infographic, 2022‐23 School Year. Lexia, A Cambium Learning Group Company.</bibtext> </blist> <blist> <bibtext> Lexia 2023b. Product Evidence Base: Lexia Core5 Reading Efficacy Research. Lexia, A Cambium Learning Group Company.</bibtext> </blist> <blist> <bibtext> Lexia 2023c. Core5 Logic Model. Lexia, A Cambium Learning Group Company.</bibtext> </blist> <blist> <bibtext> Li, S., and W. Wang. 2022. " Effect of Blended Learning on Student Performance In K‐12 Settings: A Meta‐Analysis." Journal of Computer Assisted Learning 38, no. 5 : 1254 – 1272. https://doi.org/10.1111/jcal.12696.</bibtext> </blist> <blist> <bibtext> Lyon, A. R., C. R. Cook, E. C. Brown, et al. 2018. " Assessing Organizational Implementation Context In the Education Sector: Confirmatory Factor Analysis of Measures of Implementation Leadership, Climate, and Citizenship." Implementation Science 13, no. 5 : 5.</bibtext> </blist> <blist> <bibtext> Lyon, A. R., C. M. Corbin, E. C. Brown, et al. 2022. " Leading the Charge In the Education Sector: Development and Validation of the School Implementation Leadership Scale (SILS)." Implementation science: IS 17, no. 48 : 48. https://doi.org/10.1186/s13012-022-01222-7.</bibtext> </blist> <blist> <bibtext> Maguire, M., and B. Delahunt. 2017. " Doing a Thematic Analysis: A Practical, Step‐by‐Step Guide for Learning and Teaching Scholars." All Ireland Journal of Higher Education 9, no. 3 : 3351. https://ojs.aishe.org/index.php/aishe-j/article/view/335.</bibtext> </blist> <blist> <bibtext> Meidelina, O., A. Y. Saleh, C. A. Cathlin, and S. A. Winesa. 2023. " Transformational Leadership and Teacher Well‐Being: A Systematic Review." Journal of Education and Learning 17, no. 3 : 417 – 424. https://doi.org/10.11591/edulearn.v17i3.20858.</bibtext> </blist> <blist> <bibtext> Meyers, A. B., J. Meyers, E. C. Graybill, S. L. Proctor, and L. Huddleston. 2012. " Ecological Approaches to Organizational Consultation and Systems Change in Educational Settings." Journal of Educational and Psychological Consultation 22, no. 1–2 : 106 – 124.</bibtext> </blist> <blist> <bibtext> Meyers, J., A. B. Meyers, S. L. Proctor, and E. C. Graybill. 2009. " Organizational Consultation and Systems Intervention." In Handbook of School Psychology edited by T. B. Gutkin and C. R. Reynolds, 921 – 940. John Wiley & Sons Inc.</bibtext> </blist> <blist> <bibtext> Miles, M. B., and A. M. Huberman. 1994. Qualitative Data Analysis: An Expanded Sourcebook. Sage.</bibtext> </blist> <blist> <bibtext> Mohammed, S. 2017. " Special Issue: Connecting Research and Practice to Understand Efficacy in K‐12 Blended Learning." Journal of Online Learning Research 3, no. 1 : 1 – 4. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). Retrieved November 7, 2023 from https://<ulink href="http://www.learntechlib.org/primary/p/177971/">www.learntechlib.org/primary/p/177971/</ulink>.</bibtext> </blist> <blist> <bibtext> National Association of School Psychologists. 2010. " National Association of School Psychologists Model for Comprehensive and Integrated School Psychological Services." School Psychology Review 39, no. 2 : 320 – 333. https://doi.org/10.1080/02796015.2010.12087783.</bibtext> </blist> <blist> <bibtext> National Association of School Psychologists 2020. The professional standards of the National Association of School Psychologists. Retrieved from https://<ulink href="http://www.nasponline.org/x55315.xml">www.nasponline.org/x55315.xml</ulink>.</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics (NCES). Organization for Economic Cooperation and Development (OECD), Program for the International Assessment of Adult Competencies (PIAAC). 2012–2015.</bibtext> </blist> <blist> <bibtext> National Reading Panel (U.S.) & National Institute of Child Health and Human Development (U.S.). 2000. Report of the National Reading Panel: Teaching Children to Read: An Evidence‐Based Assessment of the Scientific Research Literature on Reading and Its Implications for Reading Instruction. U.S. Dept. of Health and Human Services, Public Health Service, National Institutes of Health, National Institute of Child Health and Human Development.</bibtext> </blist> <blist> <bibtext> Nederhof, A. J. 1985. " Methods of Coping With Social Desirability Bias: A Review." European Journal of Social Psychology 15, no. 3 : 263 – 280. https://doi.org/10.1002/ejsp.2420150303.</bibtext> </blist> <blist> <bibtext> Newell, M., C. Tindall, K. Reed, and S. Zwolski. 2020. " School‐Based Consultation." In Consultation in Psychology: A Competency‐Based Approach edited by C. A. Falender and E. P. Shafranske, 201 – 219. American Psychological Association.</bibtext> </blist> <blist> <bibtext> Newman, D. S., C. A. Barrett, and C. E. Hazel. 2015. " School Consultation Practices in the Early Career: Does Training Matter? " Consulting Psychology Journal: Practice and Research 67 : 326 – 347. https://doi.org/10.1037/cpb0000048.</bibtext> </blist> <blist> <bibtext> O'Callaghan, P., A. McIvor, C. McVeigh, and T. Rushe. 2016. " A Randomized Controlled Trial of an Early Intervention, Computer‐Based Literacy Program to Boost Phonological Skills in 4‐ to 6‐Year Old Children." British Journal of Educational Psychology 86, no. 4 : 546 – 558. https://doi.org/10.1111/bjep.12122.</bibtext> </blist> <blist> <bibtext> Palinkas, L. A., G. A. Aarons, S. Horwitz, P. Chamberlain, M. Hurlburt, and J. Landsverk. 2011. " Mixed Method Designs in Implementation Research." Administration and Policy in Mental Health and Mental Health Services Research 38, no. 1 : 44 – 53. https://doi.org/10.1007/s10488-010-0314-z.</bibtext> </blist> <blist> <bibtext> Perfetti, C. A. 1985. Reading Ability. Oxford University Press.</bibtext> </blist> <blist> <bibtext> Powell, A., J. Watson, and P. Staley, et al. 2015. Blended Learning: The Evolution of Online and Face‐to‐Face Education From 2008–2015. Promising Practices in Online Learning. International Association for K–12 Online Learning.</bibtext> </blist> <blist> <bibtext> Prescott, J. E., K. Bundschuh, E. R. Kazakoff, and P. Macaruso. 2017. " Elementary School–Wide Implementation of a Blended Learning Program for Reading Intervention." The Journal of Educational Research 111, no. 4 : 497 – 506. https://doi.org/10.1080/00220671.2017.1302914.</bibtext> </blist> <blist> <bibtext> Pytash, K. E., and W. I. O'Byrne. 2018. " Handbook of Research on K‐12 Online and Blending Learning." In Research on Literacy Instruction and Learning in Virtual, Blended, and Hybrid Environments edited by K. Kennedy and R. E. Ferdig, 303 – 314. Carnegie Mellon University Press/ETC Press.</bibtext> </blist> <blist> <bibtext> Schechter, R., P. Macaruso, E. R. Kazakoff, and E. Brooke. 2015. " Exploration of a Blended Learning Approach to Reading Instruction for Low SES Students in Early Elementary Grades." Computers in the Schools 32 : 183 – 200. https://doi.org/10.1080/07380569.2015.1100652.</bibtext> </blist> <blist> <bibtext> Schwartz, H. L., D. Grant, M. K. Diliberti, G. P. Hunter, and C. M. Setodji. 2020. Remote Learning Is Here to Stay: Results From the First American School District Panel Survey. RAND Corporation RR‐A956‐1.</bibtext> </blist> <blist> <bibtext> U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). 2022. NAEP Report Card: Reading. United States [Web Archive]. https://<ulink href="http://www.nationsreportcard.gov/reading/nation/achievement/?grade=4">www.nationsreportcard.gov/reading/nation/achievement/?grade=4</ulink>.</bibtext> </blist> <blist> <bibtext> Wilkes, S., E. R. Kazakoff, J. E. Prescott, et al. 2020. " Measuring the Impact of a Blended Learning Model on Early Literacy Growth." Journal of Computer Assisted Learning 36, no. 5 : 595 – 609. https://doi.org/10.1111/jcal.12429.</bibtext> </blist> </ref> <aug> <p>By Cara M. Bliss and Courtenay A. Barrett</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib57" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib13" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib35" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib44" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib19" firstref="ref6"></nolink> <nolink nlid="nl6" bibid="bib54" firstref="ref12"></nolink> <nolink nlid="nl7" bibid="bib56" firstref="ref13"></nolink> <nolink nlid="nl8" bibid="bib17" firstref="ref14"></nolink> <nolink nlid="nl9" bibid="bib58" firstref="ref15"></nolink> <nolink nlid="nl10" bibid="bib33" firstref="ref18"></nolink> <nolink nlid="nl11" bibid="bib30" firstref="ref19"></nolink> <nolink nlid="nl12" bibid="bib22" firstref="ref20"></nolink> <nolink nlid="nl13" bibid="bib24" firstref="ref21"></nolink> <nolink nlid="nl14" bibid="bib51" firstref="ref22"></nolink> <nolink nlid="nl15" bibid="bib45" firstref="ref23"></nolink> <nolink nlid="nl16" bibid="bib20" firstref="ref24"></nolink> <nolink nlid="nl17" bibid="bib31" firstref="ref25"></nolink> <nolink nlid="nl18" bibid="bib26" firstref="ref26"></nolink> <nolink nlid="nl19" bibid="bib49" firstref="ref27"></nolink> <nolink nlid="nl20" bibid="bib55" firstref="ref28"></nolink> <nolink nlid="nl21" bibid="bib52" firstref="ref30"></nolink> <nolink nlid="nl22" bibid="bib18" firstref="ref32"></nolink> <nolink nlid="nl23" bibid="bib53" firstref="ref33"></nolink> <nolink nlid="nl24" bibid="bib32" firstref="ref38"></nolink> <nolink nlid="nl25" bibid="bib21" firstref="ref41"></nolink> <nolink nlid="nl26" bibid="bib37" firstref="ref42"></nolink> <nolink nlid="nl27" bibid="bib41" firstref="ref54"></nolink> <nolink nlid="nl28" bibid="bib14" firstref="ref55"></nolink> <nolink nlid="nl29" bibid="bib34" firstref="ref65"></nolink> <nolink nlid="nl30" bibid="bib47" firstref="ref68"></nolink> <nolink nlid="nl31" bibid="bib28" firstref="ref69"></nolink> <nolink nlid="nl32" bibid="bib48" firstref="ref70"></nolink> <nolink nlid="nl33" bibid="bib42" firstref="ref71"></nolink> <nolink nlid="nl34" bibid="bib29" firstref="ref74"></nolink> <nolink nlid="nl35" bibid="bib39" firstref="ref75"></nolink> <nolink nlid="nl36" bibid="bib12" firstref="ref76"></nolink> <nolink nlid="nl37" bibid="bib38" firstref="ref78"></nolink> <nolink nlid="nl38" bibid="bib15" firstref="ref81"></nolink> <nolink nlid="nl39" bibid="bib50" firstref="ref82"></nolink> <nolink nlid="nl40" bibid="bib23" firstref="ref88"></nolink> <nolink nlid="nl41" bibid="bib25" firstref="ref89"></nolink> <nolink nlid="nl42" bibid="bib11" firstref="ref91"></nolink> <nolink nlid="nl43" bibid="bib10" firstref="ref105"></nolink> <nolink nlid="nl44" bibid="bib36" firstref="ref106"></nolink> <nolink nlid="nl45" bibid="bib40" firstref="ref110"></nolink> <nolink nlid="nl46" bibid="bib27" firstref="ref138"></nolink> <nolink nlid="nl47" bibid="bib16" firstref="ref139"></nolink> <nolink nlid="nl48" bibid="bib46" firstref="ref141"></nolink> <nolink nlid="nl49" bibid="bib43" firstref="ref146"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1477327
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Systems-Level Consultation to Improve Intervention Fidelity and Student Outcomes: A Collective Case Study of a Blended Learning Reading Program
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Cara+M%2E+Bliss%22">Cara M. Bliss</searchLink> (ORCID <externalLink term="http://orcid.org/0009-0008-8866-0631">0009-0008-8866-0631</externalLink>)<br /><searchLink fieldCode="AR" term="%22Courtenay+A%2E+Barrett%22">Courtenay A. Barrett</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-9258-4725">0000-0001-9258-4725</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Psychology+in+the+Schools%22"><i>Psychology in the Schools</i></searchLink>. 2025 62(8):2579-2593.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Blended+Learning%22">Blended Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Programs%22">Reading Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Fidelity%22">Fidelity</searchLink><br /><searchLink fieldCode="DE" term="%22Consultation+Programs%22">Consultation Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Implementation%22">Program Implementation</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Skills%22">Reading Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Systems+Approach%22">Systems Approach</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/pits.23490
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0033-3085<br />1520-6807
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: School psychologists have the potential to improve students' reading skills by helping schools select evidence-based programs and then supporting their implementation. Blended learning offers a flexible implementation format to address long-standing challenges implementing and scaling reading programs in schools. The purpose of this study was to explore the unique implementation challenges associated with blended learning programs, how systems-level consultation can be used to support implementation and foster intervention fidelity, and the extent to which intervention fidelity of a blended learning reading program was associated with student reading skills. This collective case study included qualitative and quantitative data from 20 schools, 666 educators, and 6208 students in the U.S. We employed mixed methods, which included inductive thematic analysis of 47 implementation plans and means difference testing to examine differences in educator engagement, student-level intervention fidelity, and student outcomes across districts with varying levels of implementation supports. Results identified several challenges encountered throughout implementation and the types of strategies employed to address those challenges. Further, results suggested that there were greater levels of fidelity and more positive student outcomes in schools accessing systems-level consultation to support implementation.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1477327
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1477327
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/pits.23490
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 2579
    Subjects:
      – SubjectFull: Blended Learning
        Type: general
      – SubjectFull: Reading Programs
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Fidelity
        Type: general
      – SubjectFull: Consultation Programs
        Type: general
      – SubjectFull: Program Implementation
        Type: general
      – SubjectFull: Reading Skills
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Systems Approach
        Type: general
      – SubjectFull: Elementary School Students
        Type: general
    Titles:
      – TitleFull: Systems-Level Consultation to Improve Intervention Fidelity and Student Outcomes: A Collective Case Study of a Blended Learning Reading Program
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Cara M. Bliss
      – PersonEntity:
          Name:
            NameFull: Courtenay A. Barrett
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0033-3085
            – Type: issn-electronic
              Value: 1520-6807
          Numbering:
            – Type: volume
              Value: 62
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: Psychology in the Schools
              Type: main
ResultId 1