View in EDS HTML Full Text PDF Full Text

Training and Technical Assistance Increase the Fidelity of Implementation of a Universal Prevention Initiative in Rural Schools: Results from a 3-Year Cluster-Randomized Trial

Saved in:
Bibliographic Details
Title: Training and Technical Assistance Increase the Fidelity of Implementation of a Universal Prevention Initiative in Rural Schools: Results from a 3-Year Cluster-Randomized Trial
Language: English
Authors: Hannah G. Calvert (ORCID 0000-0002-6067-693X), Christopher M. Fleming, Michaela Lowe, Teri Lewis, Carl F. Siebert, Ashley Havlicak, Nate Anderson, Tate Castleton, Lindsey Turner
Source: Prevention Science. 2025 26(1):56-68.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 13
Publication Date: 2025
Sponsoring Agency: National Institute of Justice (NIJ) (DOJ)
Contract Number: 2017CKBX0021
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Education
Secondary Education
Descriptors: Elementary School Students, Secondary School Students, Positive Behavior Supports, Student Behavior, Intervention, Rural Schools, Evidence Based Practice, Leadership Training, Professional Development, Resource Allocation, Technical Assistance, Electronic Learning, Workshops, Achievement Gains, Educational Environment, Summer Programs, Fidelity, Prevention
DOI: 10.1007/s11121-025-01776-0
ISSN: 1389-4986
1573-6695
Abstract: The need for well-implemented evidence-based interventions (EBIs) for the prevention of behavioral issues among children and adolescents is substantial. In rural areas, the need often matches or surpasses that of urban areas. Schools have a wide reach for prevention-focused EBIs. However, implementation in rural schools is often hindered by limited resources and capacity. Rural School Support Strategies (RS3) are a bundle of implementation supports that address implementation challenges in rural settings. They include providing additional leadership and coaching training, individualized technical assistance (mostly virtual), and monthly meetings of a virtual learning collaborative. A cluster-randomized Hybrid Type 3 implementation-effectiveness trial tested RS3 for implementing school-wide positive behavioral interventions and supports (PBIS), a universal prevention approach to improving student behavior, academic outcomes, and school climate. Forty rural schools received a multi-day training on PBIS each summer for 3 years. Half were randomized to also receive RS3 support. Linear and logistic regression models examined the effect of treatment condition and dosage of support on implementation fidelity for Tier 1 (universal) PBIS. Condition and dosage (number of hours) of support increased the odds of schools achieving the 70% threshold for adequate implementation fidelity. In the first year, the higher dosage of technical assistance events increased the likelihood of schools reaching fidelity, whereas later in the trial, the higher dosage of attendance at the virtual learning collaborative sessions yielded significant improvements in fidelity. Implications for accelerating the implementation of universal prevention initiatives in schools--particularly in rural settings--are discussed. This study was prospectively registered on ClinicalTrials.gov (NCT03736395), on November 9, 2018.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1460850
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFWe8B4BP1pbeqvZDlkW1QrAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDPRxJ2fyGGCLIT82DAIBEICBmiyHJDBh_zfyFjmF379pBRSJnA46i3KjB_3LmtzwKhEOFGDtUrNxryc1aOl7ZAxdY1_ON67IL3chDRLXnKrwcNsRTgRKpBXM5bmaB0mBkOlBG18igd-iei4Alr94_7oLVdDROJA_itLWfWfi3OKWwajewaFmStNGm59DDEGxOPNx2LmpBcKrc6SWV-ipeT1_wSF4v7vTtorP_L0=
Text:
  Availability: 1
  Value: <anid>AN0182958062;n9p01jan.25;2025Feb13.05:21;v2.2.500</anid> <title id="AN0182958062-1">Training and Technical Assistance Increase the Fidelity of Implementation of a Universal Prevention Initiative in Rural Schools: Results from a 3-Year Cluster-Randomized Trial </title> <p>The need for well-implemented evidence-based interventions (EBIs) for the prevention of behavioral issues among children and adolescents is substantial. In rural areas, the need often matches or surpasses that of urban areas. Schools have a wide reach for prevention-focused EBIs. However, implementation in rural schools is often hindered by limited resources and capacity. Rural School Support Strategies (RS3) are a bundle of implementation supports that address implementation challenges in rural settings. They include providing additional leadership and coaching training, individualized technical assistance (mostly virtual), and monthly meetings of a virtual learning collaborative. A cluster-randomized Hybrid Type 3 implementation-effectiveness trial tested RS3 for implementing school-wide positive behavioral interventions and supports (PBIS), a universal prevention approach to improving student behavior, academic outcomes, and school climate. Forty rural schools received a multi-day training on PBIS each summer for 3 years. Half were randomized to also receive RS3 support. Linear and logistic regression models examined the effect of treatment condition and dosage of support on implementation fidelity for Tier 1 (universal) PBIS. Condition and dosage (number of hours) of support increased the odds of schools achieving the 70% threshold for adequate implementation fidelity. In the first year, the higher dosage of technical assistance events increased the likelihood of schools reaching fidelity, whereas later in the trial, the higher dosage of attendance at the virtual learning collaborative sessions yielded significant improvements in fidelity. Implications for accelerating the implementation of universal prevention initiatives in schools—particularly in rural settings—are discussed. This study was prospectively registered on ClinicalTrials.gov (NCT03736395), on November 9, 2018.</p> <p>Keywords: Technical assistance; Virtual learning; Intervention; Implementation; Health; Education Specialist Studies In Education</p> <p>Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11121-025-01776-0.</p> <hd id="AN0182958062-2">Introduction</hd> <p>There is a need for well-implemented evidence-based interventions (EBIs) for the prevention of behavioral issues among children and adolescents. In rural areas, the need often matches or surpasses that in urban areas, given that children and adolescents in rural areas experience similar or greater levels of behavioral health issues including substance use (Lambert et al., [<reflink idref="bib33" id="ref1">33</reflink>]), justice system involvement (Blackmon et al., [<reflink idref="bib6" id="ref2">6</reflink>]), anxiety, and depression (Figas et al., [<reflink idref="bib24" id="ref3">24</reflink>]). Problem behaviors often share common malleable risk and protective factors, and schools are an important setting for establishing safe and nurturing social environments for children and youth. Furthermore, in many rural communities, schools are a central hub for services. Universal prevention programming in schools is important for supporting rural students' behavioral and mental health needs (Hoover & Mayworm, [<reflink idref="bib30" id="ref4">30</reflink>]).</p> <p>Despite the importance of school-based universal prevention interventions, rural communities face several obstacles to the implementation of EBIs. In rural schools, the implementation of new initiatives is often hindered by limited resources and capacity. In addition to school-level factors, such as leadership and staff capacity, policies and resources at the local and state levels can all contribute to whether scale-up is successful (Fagan et al., [<reflink idref="bib23" id="ref5">23</reflink>]). There is expanding interest in leveraging implementation science to enhance the scale-up of EBIs in schools (Bradshaw, [<reflink idref="bib7" id="ref6">7</reflink>]; Lyon & Bruns, [<reflink idref="bib36" id="ref7">36</reflink>]). Similar to implementation approaches in healthcare and public health (e.g., Durlak & DuPre, [<reflink idref="bib22" id="ref8">22</reflink>]; Wandersman et al., [<reflink idref="bib58" id="ref9">58</reflink>]), strategies to increase school-level capacity for the implementation of EBIs include both pre-implementation training and other supports during the implementation process, such as technical assistance (TA; Domitrovich et al., [<reflink idref="bib20" id="ref10">20</reflink>]; Pas et al., [<reflink idref="bib50" id="ref11">50</reflink>]). TA is an individualized approach that builds organizational capacity to implement EBIs (Chinman et al., [<reflink idref="bib15" id="ref12">15</reflink>]). Although TA is highly effective for supporting organizations during the implementation of complex interventions (Wandersman et al., [<reflink idref="bib58" id="ref13">58</reflink>]), its core features, including timing, dose, and types of implementation strategies used during TA meetings, are still under investigation (Katz & Wandersman, [<reflink idref="bib32" id="ref14">32</reflink>]; Le et al., [<reflink idref="bib34" id="ref15">34</reflink>]). Learning collaboratives, which allow practitioners to engage (virtually or in-person) with one another as a mutual capacity-building exercise, may also help improve implementation outcomes in school mental health systems (Orenstein et al., [<reflink idref="bib48" id="ref16">48</reflink>]; Zubkoff et al., [<reflink idref="bib61" id="ref17">61</reflink>]).</p> <hd id="AN0182958062-3">School-Wide Positive Behavioral Interventions and Supports (PBIS)</hd> <p>PBIS is an evidence-based, multi-tiered framework to support students' well-being and academic learning (Sugai & Horner, [<reflink idref="bib54" id="ref18">54</reflink>]). Several randomized trials have shown that PBIS is effective in improving overall school climate (Bradshaw et al., [<reflink idref="bib8" id="ref19">8</reflink>], [<reflink idref="bib9" id="ref20">9</reflink>]), as well as the academic and behavioral outcomes of students (Bradshaw et al., [<reflink idref="bib11" id="ref21">11</reflink>]; Horner et al., [<reflink idref="bib31" id="ref22">31</reflink>]; Noltemeyer et al., [<reflink idref="bib46" id="ref23">46</reflink>]; Pas et al., [<reflink idref="bib51" id="ref24">51</reflink>]). PBIS promotes safe and effective school environments by establishing a coordinated system for teaching and acknowledging appropriate behavior and establishing predictable consequences for problematic behaviors. As with many other prevention initiatives (Domitrovich et al., [<reflink idref="bib20" id="ref25">20</reflink>]; Hoagwood & Johnson, [<reflink idref="bib29" id="ref26">29</reflink>]), PBIS uses a three-tiered continuum of support, consistent with a public health and prevention science approach, to reduce risk factors and increase protective factors to prevent the progression of problematic behaviors and academic underperformance. At Tier 1, efforts involve supporting all students and changing the whole school environment. Advanced tiers (Tiers 2 and 3) provide targeted and intensive interventions for students who need additional support (Sugai & Horner, [<reflink idref="bib54" id="ref27">54</reflink>]).</p> <p>PBIS is disproportionally adopted in urban and suburban schools compared to rural schools. Rural schools represent about 20% of schools implementing PBIS (Chaparro et al., [<reflink idref="bib13" id="ref28">13</reflink>]) despite making up 32% of all public schools (National Center for Education Statistics, [<reflink idref="bib44" id="ref29">44</reflink>]). Geographic distance and funding constraints pose challenges for access to professional development and TA. However, rural communities also have unique strengths that make universal programming effective, including smaller schools with lower student-to-staff ratios (Molloy et al., [<reflink idref="bib43" id="ref30">43</reflink>]), close-knit communities, innovative problem solving, and community pride (McDaniel & Bloomfield, [<reflink idref="bib38" id="ref31">38</reflink>]). Implementation science methods and frameworks can answer questions about how to improve the scale-up of EBIs in schools, particularly in rural areas.</p> <hd id="AN0182958062-4">Study Purpose</hd> <p>Few RCTs have been conducted investigating strategies for improving the implementation of universal EBIs in schools (Baffsky et al., [<reflink idref="bib3" id="ref32">3</reflink>], [<reflink idref="bib4" id="ref33">4</reflink>]). The few that have been conducted show that training combined with consultation and coaching (i.e., TA) are effective strategies for improving implementation fidelity (Becker et al., [<reflink idref="bib5" id="ref34">5</reflink>]; Bradshaw et al., [<reflink idref="bib8" id="ref35">8</reflink>], [<reflink idref="bib10" id="ref36">10</reflink>]) and similar results have been found in non-randomized trials (McDaniel & Bloomfield, [<reflink idref="bib37" id="ref37">37</reflink>]; von der Embse et al., [<reflink idref="bib57" id="ref38">57</reflink>]). However, the literature on the relative effectiveness and dosage of different types of capacity-building strategies for EBIs in schools is underdeveloped, particularly for rural schools (Cook et al., [<reflink idref="bib17" id="ref39">17</reflink>]; Domitrovich et al., [<reflink idref="bib21" id="ref40">21</reflink>]). The purpose of the current Hybrid Type 3 trial was to examine the relative effectiveness of a bundle of implementation supports called Rural School Support Strategies (RS3; Turner et al., [<reflink idref="bib56" id="ref41">56</reflink>]) on school-level implementation of a universal prevention initiative in rural schools, over 3 years. The primary outcome was the fidelity of PBIS implementation at Tier 1. Analyses explored the following research questions: (<reflink idref="bib1" id="ref42">1</reflink>) To what extent do the implementation supports, overall and individually by type, improve schools' implementation fidelity outcomes each year? (<reflink idref="bib2" id="ref43">2</reflink>) Do the implementation supports, overall or individually by type, make schools more likely to achieve 70% implementation fidelity? We hypothesized that—relative to schools receiving training only—the schools receiving RS3 supports would show (<reflink idref="bib1" id="ref44">1</reflink>) significantly greater increases in fidelity and (<reflink idref="bib2" id="ref45">2</reflink>) would be more likely to reach the 70% threshold considered sufficient for fidelity. Exploratory analyses examined how the dosage of RS3 components affected fidelity.</p> <hd id="AN0182958062-5">Methods</hd> <p>The trial took place during the 2019–2020, 2020–2021, and 2021–2022 school years. All 40 participating schools received basic support for PBIS implementation, which included guidance in assembling a building-level implementation team (five to eight school staff, including an administrator) and yearly team trainings on the PBIS framework. The 20 schools randomized to the intervention received the RS3 supports in addition to training (described below). The trial tested a "blended strategy" (Proctor et al., [<reflink idref="bib53" id="ref46">53</reflink>]) relative to active "treatment as usual." We describe the arms of the trial as "basic support" or "RS3" rather than as control and intervention.</p> <p>Hybrid trial designs are used to rigorously examine interventions to improve the implementation of EBIs (Curran et al., [<reflink idref="bib18" id="ref47">18</reflink>]). A cluster-randomized Hybrid Type 3 design primarily examines strategies and implementation outcomes at the organization level while also testing individual-level outcomes relevant to the EBI being implemented. The original trial was designed to examine student-level outcomes including behavior, perceived climate, and academic outcomes (i.e., individual-level measures of the clinical effectiveness of the EBI); however, due to pandemic-related disruptions to education systems, these measures could not be collected during the trial. Consequently, analyses examined only organizational-level implementation outcomes (i.e., fidelity). The study protocol is described in detail elsewhere (Turner et al., [<reflink idref="bib56" id="ref48">56</reflink>]), with a description of pandemic-related impacts using the CONSERVE (CONSORT and SPIRIT Extension for RCTs Revised in Extenuating Circumstances) guidelines (Orkin et al., [<reflink idref="bib49" id="ref49">49</reflink>]). The trial was intended to occur over 2 years, but due to the pandemic, a funding extension was granted, allowing the intervention to continue for a third year. Two of the 40 schools declined to continue for the optional extension.</p> <hd id="AN0182958062-6">Theoretical Framework</hd> <p>This study of RS3 was based on the Interactive Systems Framework for Dissemination and Implementation (ISF; Wandersman et al., [<reflink idref="bib59" id="ref50">59</reflink>]) which specifies three systems that interact to support the implementation of evidence-based prevention practice. The research group served as (<reflink idref="bib1" id="ref51">1</reflink>) the synthesis and translation system (distilling evidence into actionable strategies for the deliverers) and (<reflink idref="bib2" id="ref52">2</reflink>) the prevention support system (providing deliverers with technical assistance throughout implementation), and schools served as (<reflink idref="bib3" id="ref53">3</reflink>) the delivery system. The Quality Implementation Framework (Meyers et al., [<reflink idref="bib42" id="ref54">42</reflink>]) details actions to be taken by each of the three parts of the ISF system during four phases of implementation: (<reflink idref="bib1" id="ref55">1</reflink>) assess context and build capacity; (<reflink idref="bib2" id="ref56">2</reflink>) create a structure for implementation, such as teams; (<reflink idref="bib3" id="ref57">3</reflink>) provide ongoing structure as implementation occurs; and (<reflink idref="bib4" id="ref58">4</reflink>) improve future applications. We used both frameworks to conceptualize the design of the trial, the RS3 intervention, the process measures, and the philosophy to guide how our team interacted with educators (i.e., the delivery systems).</p> <hd id="AN0182958062-7">Basic Supports and Rural School Support Strategies (RS3)</hd> <p></p> <hd id="AN0182958062-8">Teaming</hd> <p>All schools were guided in developing a PBIS implementation team of five to eight people, including the school principal and one coach who worked together to guide the team.</p> <hd id="AN0182958062-9">Yearly PBIS Trainings</hd> <p>All schools were provided trainings on PBIS content in the summers of 2019, 2020, and 2021. The content was aligned with PBIS Tiers, with Year 1 covering Tier 1 (universal prevention), Year 2 covering Tier 2 (targeted interventions) plus a refresher on Tier 1, and Year 3 covering Tier 3 (intensive) interventions with a Tier 1 refresher and integration of practices across all tiers. Training was conducted in-person in 2019 and virtually in 2020 and 2021. All members of each school's implementation team attended the trainings. Schools were grouped by treatment condition and geographic region. Activities included didactic lectures, full-group problem solving, and time for teams to collaborate and build their action plan. Trainings lasted 3–4 days and occurred across seven different locations (for in-person trainings in year 1; and regional groupings were maintained for the subsequent virtual trainings) over the course of seven weeks during June and July.</p> <hd id="AN0182958062-10">Technical Assistance (TA)</hd> <p>Proactive in-person and remote TA was delivered on a monthly basis to RS3 schools by two implementation support practitioners (ISPs; Albers et al., [<reflink idref="bib1" id="ref59">1</reflink>]; Metz et al., [<reflink idref="bib41" id="ref60">41</reflink>]). The ISPs were K-12 educators with strong content expertise and credibility, having previously worked in rural Idaho schools leading PBIS implementation. During the first 3 months of the trial, ISPs traveled to the RS3 schools for in-person TA visits. In the winter of 2019, visits transitioned to a virtual format (Zoom) due to the hazards of traveling to remote mountainous regions in winter. The trial design planned a mix of TA provided primarily through a virtual modality with occasional onsite TA visits. However, the COVID-19 pandemic forced all TA to be delivered virtually after the first 3 months. In the third year of the trial, supplemental TA was provided by an additional ISP with expertise in advanced tiers of PBIS (targeted and intensive interventions for selected students).</p> <p>TA was tailored to each school's needs, generally including guidance on data-based decision-making, problem solving, following their action plan, and engaging in audit and feedback. Typically, each school's assigned ISP would attend the team's monthly meeting (in-person at first, then virtually), and coaches could email or call their ISP. Although ISPs proactively contacted RS3 coaches every month, each school varied in the amount of TA used.</p> <hd id="AN0182958062-11">Virtual Learning Collaborative (VLC)</hd> <p>The ISPs hosted monthly meetings of a virtual learning collaborative (VLC; Zubkoff et al., [<reflink idref="bib61" id="ref61">61</reflink>]) for RS3 coaches. Each hour-long session began with a presentation on a specific topic (e.g., coaching strategies, PBIS refreshers, wellness). In the second half hour, coaches were given time to share experiences, solve problems, and ask the ISPs questions. Attendance averaged 12 coaches per meeting.</p> <hd id="AN0182958062-12">Additional Year 1 Didactic Trainings</hd> <p>The coach and administrator at RS3 schools received three additional in-person trainings in the first year. The first focused on planning for implementation. The second two focused on coaching school teams through implementation. Each training lasted 1 full day and was attended by all RS3 schools. As there was no variability in attendance, this element of RS3 was not included in dosage analyses.</p> <hd id="AN0182958062-13">Online Resources</hd> <p>RS3 schools had access to a password-protected web portal with resources (e.g., videos, forms). Log-in data showed that utilization was low, mainly occurring after the ISPs had referred coaches to resources during TA meetings, thus dosage was not included in analyses as it was already reflected by the TA calculations.</p> <hd id="AN0182958062-14">Sample</hd> <p>Demographic characteristics of all rural, public K-12 schools in Idaho were obtained from the Common Core of Data (National Center for Education Statistics, [<reflink idref="bib45" id="ref62">45</reflink>]). In the fall of 2018, all public schools serving any grade level from Kindergarten to Grade 12 were invited to apply for the trial if they were located in a rural area or township, had at least 100 students, and had no prior PBIS training. Invitations were sent via mail and email and were followed up by phone calls. Forty schools were recruited. This sample size was planned to allow sufficient power (> 0.80) to detect a small-to-medium (<emph>d</emph> = 0.30) effect size in a school-level regression model. Once 40 schools were recruited (three additional schools were waitlisted) and each school's principal and district administrator completed a memorandum of understanding and consent to participate, the schools were pair-matched by size (number of students), poverty level (percent of students eligible for free or reduced-price lunches), and grades served. Pair-matching also accounted for district membership. One school (or district, for district-level blocking) within each pair was randomized to receive RS3 and the other to basic support, via coin flip. The randomization procedure was conducted by project staff and observed by three independent researchers including a doctoral-level statistician. Chi-square and <emph>t</emph>-tests confirmed that demographic characteristics (number of teachers, number of students, and percentage of students eligible for free/reduced-price meals) were not significantly different between groups. Generally, schools were small, with just under 350 students and 19 teachers, on average. Most were rural remote (42.5%; more than 25 miles from a metropolitan area) or rural distant (40%; 5 to 25 miles from a metropolitan area). Most were K-5 or K-8 schools (57.5%), with 20% middle schools, 10% high schools, and 12.5% serving all grades K-12. The CONSORT diagram is shown in Fig. 1.</p> <p>Graph: Fig. 1 CONSORT diagram</p> <p>Consistent with the racial demographics of rural Idaho, schools served predominantly White student populations (range = 33–97%; <emph>M</emph> = 77%, SD = 16%), with substantial Hispanic (<emph>M</emph> = 19%, SD = 16%) representation, and other races reported at ≤ 2% on average. Socioeconomic characteristics of the communities served by each school varied, as proxied by the percentage of students eligible for free/reduced-priced school meals. To maintain school anonymity, this indicator is reported by collapsing into three groups: 35% of schools had < 40% eligibility (lower poverty), 42.5% had 40–59% eligibility, and 22.5% had ≥ 60% eligibility (higher poverty).</p> <hd id="AN0182958062-15">Measures</hd> <p></p> <hd id="AN0182958062-16">Dosage of Implementation Support</hd> <p>Attendance at trainings, TA events, and VLC sessions was documented by the project team. Trainings and VLC sessions had a set duration for each occurrence (8 h per training and 1 h per VLC session). Individual TA meetings were variable. Each TA event was documented by the ISPs using an online tracking log including the type of meeting (in-person, zoom, or phone), recipient(s), duration, and notes on the content and topic of the meeting. Five VLC sessions were held in 2019–2020, and nine occurred each year thereafter (VLC sessions were not offered monthly as intended in spring 2020 due to pandemic-related disruptions). The dosage of support was calculated for each school, conceptualized as the number of hours of implementation support, both within type and total. Table 1 shows descriptive statistics for implementation support events for RS3 schools.</p> <p>Table 1 Dosage (number of hours) of implementation supports; RS3 schools only (<emph>n</emph> = 20)</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Year and type</p></th><th align="left" colspan="4"><p>Dosage (number of hours)</p></th></tr><tr><th align="left"><p>Min</p></th><th align="left"><p>Max</p></th><th align="left"><p>Mean</p></th><th align="left"><p>SD</p></th></tr></thead><tbody><tr><td align="left" colspan="5"><p><bold>2019–2020</bold></p></td></tr><tr><td align="left"><p>Additional trainings</p></td><td char="." align="char"><p>24.0</p></td><td char="." align="char"><p>24.0</p></td><td char="." align="char"><p>24.0</p></td><td char="." align="char"><p>0.0</p></td></tr><tr><td align="left"><p>VLC sessions</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>5.0</p></td><td char="." align="char"><p>3.7</p></td><td char="." align="char"><p>1.6</p></td></tr><tr><td align="left"><p>TA events (total)</p></td><td char="." align="char"><p>4.0</p></td><td char="." align="char"><p>34.0</p></td><td char="." align="char"><p>10.1</p></td><td char="." align="char"><p>7.0</p></td></tr><tr><td align="left"><p><italic>In-person</italic></p></td><td char="." align="char"><p>2.5</p></td><td char="." align="char"><p>28.0</p></td><td char="." align="char"><p>7.2</p></td><td char="." align="char"><p>6.2</p></td></tr><tr><td align="left"><p><italic>Virtual</italic></p></td><td char="." align="char"><p>1.0</p></td><td char="." align="char"><p>6.0</p></td><td char="." align="char"><p>2.7</p></td><td char="." align="char"><p>1.3</p></td></tr><tr><td align="left"><p><italic>Phone</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>3.0</p></td><td char="." align="char"><p>0.3</p></td><td char="." align="char"><p>0.7</p></td></tr><tr><td align="left"><p>Total</p></td><td char="." align="char"><p>30.0</p></td><td char="." align="char"><p>63.0</p></td><td char="." align="char"><p>37.8</p></td><td char="." align="char"><p>7.7</p></td></tr><tr><td align="left" colspan="5"><p><bold>2020–2021</bold></p></td></tr><tr><td align="left"><p>VLC sessions</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>9.0</p></td><td char="." align="char"><p>6.1</p></td><td char="." align="char"><p>3.3</p></td></tr><tr><td align="left"><p>TA events (total)</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>5.5</p></td><td char="." align="char"><p>3.5</p></td><td char="." align="char"><p>1.5</p></td></tr><tr><td align="left"><p><italic>In-person</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>0.0</p></td></tr><tr><td align="left"><p><italic>Virtual</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>5.5</p></td><td char="." align="char"><p>3.3</p></td><td char="." align="char"><p>1.5</p></td></tr><tr><td align="left"><p><italic>Phone</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>1.8</p></td><td char="." align="char"><p>0.2</p></td><td char="." align="char"><p>0.5</p></td></tr><tr><td align="left"><p>Total</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>14.3</p></td><td char="." align="char"><p>9.5</p></td><td char="." align="char"><p>4.2</p></td></tr><tr><td align="left" colspan="5"><p><bold>2021–2022</bold></p></td></tr><tr><td align="left"><p>VLC sessions</p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>9.0</p></td><td char="." align="char"><p>4.2</p></td><td char="." align="char"><p>3.0</p></td></tr><tr><td align="left"><p>TA events (total)</p></td><td char="." align="char"><p>1.0</p></td><td char="." align="char"><p>8.5</p></td><td char="." align="char"><p>4.3</p></td><td char="." align="char"><p>2.0</p></td></tr><tr><td align="left"><p><italic>In-person</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>1.0</p></td><td char="." align="char"><p>0.1</p></td><td char="." align="char"><p>0.3</p></td></tr><tr><td align="left"><p><italic>Virtual</italic></p></td><td char="." align="char"><p>1.0</p></td><td char="." align="char"><p>8.0</p></td><td char="." align="char"><p>4.1</p></td><td char="." align="char"><p>1.9</p></td></tr><tr><td align="left"><p><italic>Phone</italic></p></td><td char="." align="char"><p>0.0</p></td><td char="." align="char"><p>0.7</p></td><td char="." align="char"><p>0.1</p></td><td char="." align="char"><p>0.2</p></td></tr><tr><td align="left"><p>Total</p></td><td char="." align="char"><p>2.0</p></td><td char="." align="char"><p>14.8</p></td><td char="." align="char"><p>8.5</p></td><td char="." align="char"><p>3.5</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Note. VLC</emph> virtual learning collaborative, <emph>TA</emph> technical assistance</p> <hd id="AN0182958062-17">Implementation Fidelity</hd> <p>Fidelity was assessed with the Tiered Fidelity Inventory (TFI; Algozzine et al., [<reflink idref="bib2" id="ref63">2</reflink>]). The TFI was designed for coaches and administrators to assess the degree to which they are adopting and applying core components of PBIS. For this study, school teams completed the TFI each year during summer training institutes. The TFI has high content validity, inter-rater (ICC = 0.99) and test–retest (ICC = 0.99) reliabilities (McIntosh et al., [<reflink idref="bib39" id="ref64">39</reflink>]), and convergent validity with other Tier 1 PBIS evaluation tools (<emph>r</emph>s from 0.65 to 0.96; Mercer et al., [<reflink idref="bib40" id="ref65">40</reflink>]). Because Tier 1 practices have the most robust associations with improved behavioral outcomes (Lee & Gage, [<reflink idref="bib35" id="ref66">35</reflink>]), the present analyses focus only on Tier 1 implementation fidelity. Tier 1 of the TFI includes 15 items across three subscales (teams, implementation, evaluation), and recent literature suggests that considering fidelity at the subscale level within each tier may provide the most accurate representation of fidelity (Grasley-Boy et al., [<reflink idref="bib28" id="ref67">28</reflink>]). Activities measured by the teams subscale and the evaluation subscale were largely facilitated by the trial (i.e., expected for schools in both conditions), so we present data from the 9-item Tier 1 implementation subscale of the TFI as a standalone outcome. This subscale assesses the degree to which the school is instituting the core components of PBIS, including defining problem behaviors and expectations, and establishing a system for feedback. Each item is rated 2, 1, or 0 points for fully, partially, or not implemented. We examined the total Tier 1 TFI score (hereafter: TFI) and implementation subscale in two ways: as a percentage (0–100%) of total possible points and as a dichotomous variable indicating whether the school met or surpassed the 70% threshold, which was suggested by the tool's creators (Algozzine et al., [<reflink idref="bib2" id="ref68">2</reflink>]) and is associated with improved student outcomes (Grasley-Boy et al., [<reflink idref="bib27" id="ref69">27</reflink>]; Noltemeyer et al., [<reflink idref="bib46" id="ref70">46</reflink>]).</p> <hd id="AN0182958062-18">School Characteristics</hd> <p>Since implementation fidelity is typically higher in primary schools (i.e., Grades K-6) (Molloy et al., [<reflink idref="bib43" id="ref71">43</reflink>]), we included school level in the models as a binary covariate (K-5/K-8 schools = 1; middle/high/K-12 schools = 0).</p> <hd id="AN0182958062-19">Analysis</hd> <p>We first examined school-level fidelity using the total TFI and implementation subscale in each year, for basic support schools and for RS3 schools, using <emph>t</emph>-tests to assess whether fidelity differed significantly within-year, unadjusted, and by condition. We also examined the number of schools meeting the 70% threshold at each time point. Thereafter, descriptive statistics were used to explore the dosage of implementation support events, considering how much variability existed in the amount of exposure to the types of support each year, among RS3 schools only. To assess the impact of RS3 on fidelity, we calculated a series of multivariable regression models to explore changes in fidelity in relation to exposure to varying dosages of support during that school year. First, we used sequential linear regressions to assess the overall condition effect, testing whether the RS3 condition improved scores on the TFI and implementation subscale each year, controlling for school level and the prior year's score on the same scale. Next, linear regressions tested whether the dosage (duration of training, TA, and VLC events attended each year) predicted improvement, including the same controls. The calculation of total dosage in 2019–2020 included three trainings (adding a constant across all RS3 schools) in addition to variable dosage of TA and VLC events for each RS3 school. In subsequent years, dosage included only TA and VLC attendance because supplemental trainings were not offered. Model fit was assessed with <emph>R</emph><sups>2</sups> values and <emph>F</emph>-tests.</p> <p>Lastly, to test whether each school attained sufficient fidelity of implementation, we re-calculated each model with logistic regression, using the 70% threshold to create a binary outcome (≥ 70% = adequate fidelity). Model fit was assessed with pseudo-<emph>R</emph><sups>2</sups> (Cox & Snell) values and omnibus chi-squared tests. Analyses were conducted on a year-to-year basis rather than examining trends over time with longitudinal modeling approaches, such as growth curve modeling. This was a modification of our pre-registered protocol (Turner et al., [<reflink idref="bib56" id="ref72">56</reflink>]) and was deemed necessary due to the non-linear pattern in TFI scores over time, due to the pandemic.</p> <hd id="AN0182958062-20">Results</hd> <p>Figure 2 shows TFI and implementation subscale scores over time at all 40 schools in the study, separated by treatment condition. At baseline, RS3 schools had slightly greater TFI scores than basic support schools (<emph>t</emph>(<reflink idref="bib38" id="ref73">38</reflink>) = 1.79, <emph>p</emph> = 0.08), but none met the 70% threshold. At the end of the first year, scores were greater for both RS3 and basic support schools and were significantly greater at RS3 schools for the TFI (<emph>t</emph>(30.8) = 2.05, <emph>p</emph> < 0.05) and implementation subscale (<emph>t</emph>(<reflink idref="bib38" id="ref74">38</reflink>) = 2.30, <emph>p</emph> < 0.05) compared to basic support schools. At the end of the first year, 16 RS3 schools met the 70% threshold for total TFI, compared to 11 basic support schools (<emph>χ</emph><sups>2</sups>(<reflink idref="bib1" id="ref75">1</reflink>) = 2.85, <emph>p</emph> = 0.09), and 15 RS3 schools met the 70% threshold for implementation, compared to seven basic support schools (<emph>χ</emph><sups>2</sups>(<reflink idref="bib1" id="ref76">1</reflink>) = 6.47, <emph>p</emph> = 0.01). TFI and implementation scores declined slightly in 2021, before increasing slightly for both conditions in the final year. In 2021, fewer schools in RS3 and basic support met the 70% threshold for TFI (14 and 9, respectively) and implementation (12 and 7, respectively) than the prior year, but by 2022, most schools met this threshold for TFI (18 and 13) and implementation (15 and 11). Condition differences were not significant in either year.</p> <p>Graph: Fig. 2 Fidelity outcomes: percentage of total points, by scale and condition. Notes. Lines portray individual schools. Black lines show the scale mean value within the condition. Dashed gray line indicates a 70% threshold</p> <hd id="AN0182958062-21">Linear Regression Models to Predict Implementation Percentage Scores</hd> <p></p> <hd id="AN0182958062-22">Effect of RS3 Condition</hd> <p>Table 2 shows the main effects from a series of linear regression models testing the effects of treatment condition— and dosage of implementation support elements—on the TFI total and implementation subscale scores. The RS3 condition had a marginally significant effect on both the TFI and implementation subscale in 2020 only, corresponding with a TFI percentage score that was 8.48 points higher among RS3 schools than basic support schools and an implementation subscale score that was 8.54 points higher than basic support schools in the first year of the trial.</p> <p>Table 2 Summary of results of separate linear regression models including key predictor variables on the Tiered Fidelity Inventory Tier 1 total score and Tiered Fidelity Inventory Tier 1 implementation subscale score</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="3" /><th align="left" colspan="5"><p>Tiered Fidelity Inventory, total score</p></th><th align="left" colspan="5"><p>Implementation subscale, subscale score</p></th></tr><tr><th align="left" colspan="4"><p>Coefficients for key predictor in each model</p></th><th align="left"><p>Model fit</p></th><th align="left" colspan="4"><p>Coefficients for key predictor in each model</p></th><th align="left"><p>Model fit</p></th></tr><tr><th align="left"><p><italic>β</italic></p></th><th align="left"><p><italic>b</italic></p></th><th align="left"><p>SE</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>R</italic><sup>2</sup></p></th><th align="left"><p><italic>β</italic></p></th><th align="left"><p><italic>b</italic></p></th><th align="left"><p>SE</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>R</italic><sup>2</sup></p></th></tr></thead><tbody><tr><td align="left"><p><bold>2019–2020</bold></p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>RS3 condition</p></td><td align="left"><p><bold>.27†</bold></p></td><td align="left"><p>8.48</p></td><td align="left"><p>4.61</p></td><td align="left"><p>.074</p></td><td align="left"><p>.27</p></td><td align="left"><p><bold>.28†</bold></p></td><td align="left"><p>8.54</p></td><td align="left"><p>4.45</p></td><td align="left"><p>.063</p></td><td align="left"><p>.29</p></td></tr><tr><td align="left"><p>Total dosage</p></td><td align="left"><p>.19</p></td><td align="left"><p>0.15</p></td><td align="left"><p>0.12</p></td><td align="left"><p>.212</p></td><td align="left"><p>.24</p></td><td align="left"><p>.25</p></td><td align="left"><p>0.19</p></td><td align="left"><p>0.12</p></td><td align="left"><p>.107</p></td><td align="left"><p>.27</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td align="left"><p>.12</p></td><td align="left"><p>0.88</p></td><td align="left"><p>1.12</p></td><td align="left"><p>.436</p></td><td align="left"><p>.22</p></td><td align="left"><p>.17</p></td><td align="left"><p>1.23</p></td><td align="left"><p>1.07</p></td><td align="left"><p>.259</p></td><td align="left"><p>.25</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td align="left"><p>.03</p></td><td align="left"><p>0.07</p></td><td align="left"><p>0.35</p></td><td align="left"><p>.845</p></td><td align="left"><p>.21</p></td><td align="left"><p>.15</p></td><td align="left"><p>0.32</p></td><td align="left"><p>0.34</p></td><td align="left"><p>.342</p></td><td align="left"><p>.24</p></td></tr><tr><td align="left"><p><bold>2020–2021</bold></p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>RS3 condition</p></td><td align="left"><p>.01</p></td><td align="left"><p>0.41</p></td><td align="left"><p>5.57</p></td><td align="left"><p>.942</p></td><td align="left"><p>.36</p></td><td align="left"><p>.09</p></td><td align="left"><p>3.74</p></td><td align="left"><p>6.42</p></td><td align="left"><p>.564</p></td><td align="left"><p>.21</p></td></tr><tr><td align="left"><p>Total dosage</p></td><td align="left"><p> −.05</p></td><td align="left"><p> − 0.18</p></td><td align="left"><p>0.49</p></td><td align="left"><p>.723</p></td><td align="left"><p>.37</p></td><td align="left"><p> −.02</p></td><td align="left"><p> − 0.07</p></td><td align="left"><p>0.57</p></td><td align="left"><p>.907</p></td><td align="left"><p>.20</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td align="left"><p> −.06</p></td><td align="left"><p> − 0.31</p></td><td align="left"><p>0.73</p></td><td align="left"><p>.670</p></td><td align="left"><p>.37</p></td><td align="left"><p> −.03</p></td><td align="left"><p> − 0.18</p></td><td align="left"><p>0.84</p></td><td align="left"><p>.832</p></td><td align="left"><p>.20</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td align="left"><p> −.03</p></td><td align="left"><p> − 0.25</p></td><td align="left"><p>1.35</p></td><td align="left"><p>.854</p></td><td align="left"><p>.36</p></td><td align="left"><p>.01</p></td><td align="left"><p>0.11</p></td><td align="left"><p>1.56</p></td><td align="left"><p>.943</p></td><td align="left"><p>.20</p></td></tr><tr><td align="left"><p><bold>2021–2022</bold></p></td><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /><td align="left" /></tr><tr><td align="left"><p>RS3 condition</p></td><td align="left"><p>.12</p></td><td align="left"><p>4.06</p></td><td align="left"><p>3.81</p></td><td align="left"><p>.294</p></td><td align="left"><p>.58</p></td><td align="left"><p>.19</p></td><td align="left"><p>6.38</p></td><td align="left"><p>4.39</p></td><td align="left"><p>.155</p></td><td align="left"><p>.44</p></td></tr><tr><td align="left"><p>Total dosage</p></td><td align="left"><p>.13</p></td><td align="left"><p>0.45</p></td><td align="left"><p>0.39</p></td><td align="left"><p>.252</p></td><td align="left"><p>.59</p></td><td align="left"><p>.18</p></td><td align="left"><p>0.63</p></td><td align="left"><p>0.45</p></td><td align="left"><p>.167</p></td><td align="left"><p>.44</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td align="left"><p><bold>.21†</bold></p></td><td align="left"><p>1.22</p></td><td align="left"><p>0.61</p></td><td align="left"><p>.055</p></td><td align="left"><p>.62</p></td><td align="left"><p><bold>.25†</bold></p></td><td align="left"><p>1.42</p></td><td align="left"><p>0.71</p></td><td align="left"><p>.053</p></td><td align="left"><p>.47</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td align="left"><p>.00</p></td><td align="left"><p> − 0.01</p></td><td align="left"><p>0.75</p></td><td align="left"><p>.992</p></td><td align="left"><p>.57</p></td><td align="left"><p>.05</p></td><td align="left"><p>0.35</p></td><td align="left"><p>0.87</p></td><td align="left"><p>.690</p></td><td align="left"><p>.41</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Notes</emph>. †<emph>p </emph><.10 *<emph>p </emph><.05 **<emph>p </emph><.01 ***<emph>p </emph><.001. Model fit (<emph>F</emph>-test with 3<emph>df</emph>) was significant at <emph>p </emph><.05 or better for all models. Each model controls for two covariates: school level (K-5/K-8=1); and prior year's TFI score. <emph>TA</emph> Technical Assistance, <emph>VLC</emph> Virtual Learning Collaborative</p> <hd id="AN0182958062-23">Dosage: Number of Hours of Implementation Support</hd> <p>No significant effects were observed for the total dosage of implementation supports on either the TFI or implementation subscale. However, marginally significant effects were observed when examining the dosage of VLC sessions in 2021–2022, for both outcomes. VLC dosage models are interpreted as each hour of VLC attendance being marginally associated with increasing TFI and implementation scores by 1.22 and 1.42 points, respectively.</p> <hd id="AN0182958062-24">Logistic Regression Models to Predict Sufficient Implementation (70% Threshold)</hd> <p>Table 3 shows models exploring whether intervention components were associated with the likelihood of schools reaching the 70% threshold for fidelity of implementation. Neither the overall intervention nor the dosage of any particular components was associated with an increased likelihood of schools attaining the 70% threshold on the TFI. Several intervention components were associated with attaining 70% on the implementation subscale in 2020. There was a significant effect for the overall RS3 intervention (AOR = 6.23, <emph>p</emph> < 0.05), while VLC dosage was marginally significant (AOR = 1.34, <emph>p</emph> = 0.098). There was a significant effect for the overall dosage of implementation supports (AOR = 1.05, <emph>p</emph> < 0.05) and marginal effects for the dosage of TA (AOR = 1.19, <emph>p</emph> = 0.056). There were no significant effects in subsequent years.</p> <p>Table 3 Summary of results of separate logistic regression models including key predictor variables on Tiered Fidelity Inventory threshold of 70% fidelity</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="3" /><th align="left" colspan="5"><p>Tiered Fidelity Inventory, 70% threshold</p></th><th align="left" colspan="5"><p>Implementation subscale, 70% threshold</p></th></tr><tr><th align="left" colspan="3"><p>Coefficients for key predictor in each model</p></th><th align="left" colspan="2"><p>Overall model fit</p></th><th align="left" colspan="3"><p>Coefficients for key predictor in each model</p></th><th align="left" colspan="2"><p>Overall model fit</p></th></tr><tr><th align="left"><p>AOR</p></th><th align="left"><p>95% CI</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>R</italic><sup>2</sup></p></th><th align="left"><p><italic>χ</italic><sup>2</sup></p></th><th align="left"><p>AOR</p></th><th align="left"><p>95% CI</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>R</italic><sup>2</sup></p></th><th align="left"><p><italic>χ</italic><sup>2</sup></p></th></tr></thead><tbody><tr><td align="left" colspan="11"><p><bold>2019–2020</bold></p></td></tr><tr><td align="left"><p>RS3 condition</p></td><td char="." align="char"><p>3.42</p></td><td align="left"><p>0.70, 16.67</p></td><td align="left"><p>.128</p></td><td align="left"><p>.20</p></td><td char="." align="char"><p>8.70*</p></td><td char="." align="char"><p><bold>6.23*</bold></p></td><td align="left"><p>1.32, 29.43</p></td><td align="left"><p>.021</p></td><td align="left"><p>.26</p></td><td char="." align="char"><p>11.90**</p></td></tr><tr><td align="left"><p>Total dosage</p></td><td char="." align="char"><p>1.02</p></td><td align="left"><p>0.98, 1.06</p></td><td align="left"><p>.333</p></td><td align="left"><p>.17</p></td><td char="." align="char"><p>7.21†</p></td><td char="." align="char"><p><bold>1.05*</bold></p></td><td align="left"><p>1.01, 1.09</p></td><td align="left"><p>.025</p></td><td align="left"><p>.25</p></td><td char="." align="char"><p>11.67**</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td char="." align="char"><p>1.09</p></td><td align="left"><p>0.76, 1.55</p></td><td align="left"><p>.654</p></td><td align="left"><p>.15</p></td><td char="." align="char"><p>6.45†</p></td><td char="." align="char"><p><bold>1.34†</bold></p></td><td align="left"><p>0.95, 1.89</p></td><td align="left"><p>.098</p></td><td align="left"><p>.20</p></td><td char="." align="char"><p>8.78*</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td char="." align="char"><p>0.99</p></td><td align="left"><p>0.88, 1.11</p></td><td align="left"><p>.852</p></td><td align="left"><p>.15</p></td><td char="." align="char"><p>6.28†</p></td><td char="." align="char"><p><bold>1.19†</bold></p></td><td align="left"><p>1.00, 1.42</p></td><td align="left"><p>.056</p></td><td align="left"><p>.24</p></td><td char="." align="char"><p>10.75*</p></td></tr><tr><td align="left" colspan="11"><p><bold>2020–2021</bold></p></td></tr><tr><td align="left"><p>RS3 condition</p></td><td char="." align="char"><p>2.05</p></td><td align="left"><p>0.42, 10.02</p></td><td align="left"><p>.374</p></td><td align="left"><p>.32</p></td><td char="." align="char"><p>15.57**</p></td><td char="." align="char"><p>2.11</p></td><td align="left"><p>0.45, 9.90</p></td><td align="left"><p>.345</p></td><td align="left"><p>.30</p></td><td char="." align="char"><p>14.00**</p></td></tr><tr><td align="left"><p>Total duration</p></td><td char="." align="char"><p>1.00</p></td><td align="left"><p>0.87, 1.15</p></td><td align="left"><p>.965</p></td><td align="left"><p>.31</p></td><td char="." align="char"><p>14.77**</p></td><td char="." align="char"><p>0.98</p></td><td align="left"><p>0.85, 1.12</p></td><td align="left"><p>.724</p></td><td align="left"><p>.28</p></td><td char="." align="char"><p>13.23**</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td char="." align="char"><p>0.96</p></td><td align="left"><p>0.78, 1.18</p></td><td align="left"><p>.703</p></td><td align="left"><p>.31</p></td><td char="." align="char"><p>14.91**</p></td><td char="." align="char"><p>0.93</p></td><td align="left"><p>0.76, 1.15</p></td><td align="left"><p>.512</p></td><td align="left"><p>.29</p></td><td char="." align="char"><p>13.54**</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td char="." align="char"><p>1.12</p></td><td align="left"><p>0.76, 1.65</p></td><td align="left"><p>.557</p></td><td align="left"><p>.32</p></td><td char="." align="char"><p>15.12**</p></td><td char="." align="char"><p>1.05</p></td><td align="left"><p>0.72, 1.53</p></td><td align="left"><p>.797</p></td><td align="left"><p>.28</p></td><td char="." align="char"><p>13.17**</p></td></tr><tr><td align="left" colspan="11"><p><bold>2021–2022</bold></p></td></tr><tr><td align="left"><p>RS3 condition</p></td><td char="." align="char"><p>4.26</p></td><td align="left"><p>0.36, 50.86</p></td><td align="left"><p>.252</p></td><td align="left"><p>.36</p></td><td char="." align="char"><p>16.93***</p></td><td char="." align="char"><p>1.73</p></td><td align="left"><p>0.25, 12.24</p></td><td align="left"><p>.581</p></td><td align="left"><p>.43</p></td><td char="." align="char"><p>21.28***</p></td></tr><tr><td align="left"><p>Total dosage</p></td><td char="." align="char"><p>1.18</p></td><td align="left"><p>0.89, 1.58</p></td><td align="left"><p>.257</p></td><td align="left"><p>.36</p></td><td char="." align="char"><p>17.03***</p></td><td char="." align="char"><p>1.06</p></td><td align="left"><p>0.87, 1.28</p></td><td align="left"><p>.588</p></td><td align="left"><p>.43</p></td><td char="." align="char"><p>21.27***</p></td></tr><tr><td align="left"><p>Dosage of VLC</p></td><td char="." align="char"><p>1.64</p></td><td align="left"><p>0.69, 3.90</p></td><td align="left"><p>.261</p></td><td align="left"><p>.38</p></td><td char="." align="char"><p>18.12***</p></td><td char="." align="char"><p>1.30</p></td><td align="left"><p>0.88, 1.92</p></td><td align="left"><p>.184</p></td><td align="left"><p>.45</p></td><td char="." align="char"><p>22.97***</p></td></tr><tr><td align="left"><p>Dosage of TA</p></td><td char="." align="char"><p>1.15</p></td><td align="left"><p>0.73, 1.81</p></td><td align="left"><p>.534</p></td><td align="left"><p>.34</p></td><td char="." align="char"><p>15.86**</p></td><td char="." align="char"><p>0.93</p></td><td align="left"><p>0.66, 1.31</p></td><td align="left"><p>.677</p></td><td align="left"><p>.43</p></td><td char="." align="char"><p>21.14***</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>Notes.</emph> Each model controls for two covariates: school level (K-5/K-8 = 1); and prior year's TFI score. <emph>R</emph><sups>2</sups> values reflect Cox & Snell pseudo-<emph>R</emph><sups>2</sups>; χ<sups>2</sups> values reflect omnibus tests with 3<emph>df</emph><emph>AOR</emph> adjusted odds ratio, <emph>CI</emph> confidence interval, <emph>TA</emph> technical assistance, <emph>VLC</emph> virtual learning collaborative <sups>†</sups><emph>p</emph> <.10, *<emph>p</emph> <.05, **<emph>p</emph> <.01, ***<emph>p</emph> <.001</p> <hd id="AN0182958062-25">Discussion</hd> <p>We found that basic implementation supports—establishing an implementation team and hosting intensive, team-based training—helped rural schools build a strong foundation for the implementation of an evidence-based universal prevention initiative. However, additional supports significantly accelerated implementation fidelity. Consistent with prior literature (Becker et al., [<reflink idref="bib5" id="ref77">5</reflink>]; Bradshaw et al., [<reflink idref="bib8" id="ref78">8</reflink>], [<reflink idref="bib10" id="ref79">10</reflink>]), as well as best practice guidelines for implementing PBIS (Fixsen et al., [<reflink idref="bib25" id="ref80">25</reflink>]) and other EBIs, we found that initial training promoted increases in implementation fidelity, but schools that received a comprehensive set of supports had significantly higher fidelity in the first year and maintained higher fidelity in years 2 and 3 (although not statistically significant). As one of the few studies to systematically measure and examine the dosage of implementation supports provided over time, our results, while necessarily interpreted with the caveat of the pandemic, provide important insights for optimizing the timing and quantity of supports to help schools to implement school-wide EBIs.</p> <p>The TA provided in our study was proactive, customized, collaborative, and long-term, all of which are aspects noted by Wandersman and colleagues (2012) as important for TA effectiveness. Our results showed that TA had a significant effect on schools reaching 70% fidelity in year 1, whereby schools that utilized more TA were more likely to attain sufficient fidelity. School teams used more TA in year 1 than in years 2 and 3 (10.1, 3.5, and 4.3 h on average, respectively), while the range between minimum and maximum hours of TA use by year was 30, 5.5, and 7.5 h. This underscores the importance of providing flexible TA based on need, with a focus on intensive support during early implementation, as suggested in TA models (e.g., Le et al., [<reflink idref="bib34" id="ref81">34</reflink>]). The fact that there was not a continued significant effect of the TA in later years is also supported by growing evidence that associations between use of supports and team/coalition functioning are not always linear throughout implementation phases (Olson et al., [<reflink idref="bib47" id="ref82">47</reflink>]). Chilenski and colleagues ([<reflink idref="bib14" id="ref83">14</reflink>]) recently showed that a higher dosage of TA (measured as the frequency of contact) early on was related to worse team functioning in the early stages, but that same high dosage of early TA was related to better team functioning later in the implementation. It is likely that TA utilization (i.e., dosage) in our study was related to implementation fidelity in the first year because schools were only focused on Tier 1 during the first year, and no schools had achieved 70% implementation fidelity yet. In subsequent years, it becomes harder to interpret the relationship between utilization of TA and fidelity of Tier 1 implementation, because it is likely that some schools continued to implement well without needing much TA, whereas others struggled while using comparatively more TA. The qualitative evidence from our trial shows that school coaches and administrators found TA to be the most acceptable and appropriate form of implementation support (Calvert et al., [<reflink idref="bib12" id="ref84">12</reflink>]).</p> <p>The research examining the effectiveness of virtual learning collaboratives as a support for improving fidelity of universal prevention EBIs in schools is growing, but still relatively nascent (Baffsky et al., [<reflink idref="bib3" id="ref85">3</reflink>]). Our approach for conducting VLC sessions included elements of training (in the form of revisiting/refreshing content) and TA, while also providing an outlet for other implementation strategies. School coaches could share data, receive feedback, and problem-solve with ISPs as well as coaches from other RS3 schools. Our results showed that higher VLC attendance predicted higher odds of reaching 70% implementation fidelity in the first year, and attendance had a marginally significant linear relationship with fidelity in year three. Average rates of attendance were similar in all study years (3.7, 3.5, and 4.3 h, respectively). The fact that the VLC was the only implementation support with a relationship (albeit marginally significant) with implementation fidelity in the final year of the trial shows the strength of these scheduled and predictable virtual communities for supporting sustained fidelity years after initial implementation. Studies have shown VLCs to be feasible and effective for supporting implementation, although the discrete implementation strategies that VLCs employ can make direct comparisons of effectiveness challenging (Connors et al., [<reflink idref="bib16" id="ref86">16</reflink>]; Orenstein et al., [<reflink idref="bib48" id="ref87">48</reflink>]). Qualitative data from the trial showed that the acceptability of VLC sessions was somewhat lower than for TA; however, the VLC was received more favorably over time, due to attendees developing deeper relationships with their peers from other RS3 schools (Calvert et al., [<reflink idref="bib12" id="ref88">12</reflink>]). The potential higher acceptability and feasibility of our VLC compared to other types of peer learning groups for school personnel (e.g., Baffsky et al., [<reflink idref="bib4" id="ref89">4</reflink>]) might be due to the strategies used; our VLC was proactively organized and led by an ISP, so coaches only had to virtually "show up" to have a tailored learning opportunity and protected time to interact with their peers.</p> <p>Our findings on the effectiveness of the RS3 implementation supports align with other research on implementation strategies in school settings. A comprehensive taxonomy of implementation strategies was identified in the Expert Recommendations for Implementing Change (ERIC) project (Powell et al., [<reflink idref="bib52" id="ref90">52</reflink>]) and then further refined for school-specific applications in the SISTER (School Implementation Strategies, Translating ERIC Resources) taxonomy by Cook and colleagues (2019). A recent systematic review of empirical evidence on the effectiveness of strategies for implementation of school-based mental health EBIs identified 21 articles (five randomized trials), with most using SISTER strategies of <emph>conduct ongoing training</emph> and <emph>provide ongoing consultation/coaching</emph> (Baffsky et al., [<reflink idref="bib3" id="ref91">3</reflink>], [<reflink idref="bib4" id="ref92">4</reflink>]). Based on the results of those studies, Baffsky and colleagues identified several promising strategies, including strategies used in our project such as engaging in audit and feedback (done by our TA providers) and engaging a school leadership team. They did not find any published evidence about the effectiveness of <emph>create a professional learning collaborative</emph>. While the TA and VLC sessions were a medium for engaging in a variety of strategies (i.e., educating, coaching, auditing, organizing; see Turner et al., [<reflink idref="bib56" id="ref93">56</reflink>] for a full list of identified SISTER strategies), due to the breadth of strategies used and long timespan of the study, we were unable to collect data regarding the effectiveness of all discrete strategies used. This highlights the need for further research on the types of activities that occur during TA and VLC sessions, and how they align with SISTER strategies. Future mixed-methods studies could improve knowledge about which strategies actually work, through protocolized approaches testing the use of discrete implementation strategies (e.g., audit/feedback versus information only) during TA visits, rather than our blended approach.</p> <hd id="AN0182958062-26">Strengths and Limitations</hd> <p>This trial was developed in line with the standards of evidence of the Society for Prevention Research (Gottfredson et al., [<reflink idref="bib26" id="ref94">26</reflink>]), emphasizing rigor in the design, which included cluster-randomization, longitudinal data collection, careful documentation of intervention components (i.e., implementation supports offered and delivered), the use of an active control condition, a focus on understudied rural schools, and baseline data collection that included measures of community context. However, there are several limitations, many of which stem from pandemic-related impacts. Pertaining to the outcome of fidelity, we designed the trial with a plan to annually collect two measures of fidelity, both created by the PBIS developers and widely used in the field. In addition to the staff-reported TFI, we planned to use the Schoolwide Evaluation Tool (SET; Sugai et al., [<reflink idref="bib55" id="ref95">55</reflink>]) to assess fidelity. The SET is completed by trained observers who are masked to treatment condition, which would have allowed an independent and unbiased outcome measure. Although SET data were collected from all schools at baseline, pandemic restrictions made on-site follow-ups impossible.</p> <p>As previously noted, the planned activities of the trial—including student-level data collection—were altered due to the pandemic. While the sample size of 40 schools is substantial for an implementation study, it is insufficiently powered for complex analyses involving only organizational-level outcomes. Our study was designed to test how changes in PBIS fidelity impacted student outcomes, but student-level outcomes were not available. Recent modeling simulations (Williams et al., [<reflink idref="bib60" id="ref96">60</reflink>]) confirm that at least 40 cluster-level cases (i.e., schools) and 600 individual-level cases (i.e., students) are needed in three-level modeling to assess mediational mechanisms of change. Those simulations suggest that our original design would have been adequately powered; however, while the sample size of 40 schools is substantial for an implementation study, it is insufficiently powered for complex analyses involving only organizational-level outcomes. Further, we were not able to adjust for family-wise error inflation. Although the random allocation to condition resulted in similar demographics across condition, other covariates—including ones that we measured such as community context and resources—may have accounted for observed variation in outcomes, but we could not add more covariates without overparameterizing the models. Because this study focused on the scale-up of a universal prevention intervention in rural schools in Idaho (small schools with a majority of White students and moderate socioeconomic disadvantage), these findings may not generalize to all rural areas. It is also important to note that fidelity is an important implementation outcome for EBIs, but without adaptability, EBIs may not be effective, appropriate, or a good fit in various settings. This fidelity-adaptation dilemma is well-known in prevention science (Dane & Schneider, [<reflink idref="bib19" id="ref97">19</reflink>]) and is also an essential consideration in implementation research, particularly in experimental trials where fidelity is a key outcome. In the current study, the EBI itself is inherently adaptable, given that PBIS is a framework, and fidelity is measured by the presence of "key features" (i.e., core components). Our interventionists encouraged adaptation based on each school's context and need but such an approach may not be appropriate with other EBIs.</p> <hd id="AN0182958062-27">Conclusions</hd> <p>Intensive team-based trainings facilitate the implementation of EBIs in rural schools, but fidelity is accelerated by additional supports, including TA and learning collaboratives. We found that the impacts of TA and VLCs on fidelity were significant even though most supports were provided virtually in the first year, which shows the value of providing hybrid online and in-person support for schools implementing an EBI. A higher dosage of supports in the first year increased the odds of adequate implementation fidelity; therefore, program developers and implementation agents should ensure that in the early phases of implementation, there is a sufficient dosage of support provided to school teams.</p> <hd id="AN0182958062-28">Author Contribution</hd> <p>LT and CFS designed the research. HGC, ML, TL, AH, NA, TA, and LT participated in data acquisition. CMF performed data analysis. CMF, LT, and HGC interpreted the data and drafted and revised the manuscript.</p> <hd id="AN0182958062-29">Funding</hd> <p>This study was funded by the National Institute of Justice, #2017-CK-BX-0021.</p> <hd id="AN0182958062-30">Data Availability</hd> <p>De-identified data, codebooks, and data collection instruments will be available at https://<ulink href="http://www.icpsr.umich.edu/web/pages/NACJD/index.html">www.icpsr.umich.edu/web/pages/NACJD/index.html</ulink>.</p> <hd id="AN0182958062-31">Declarations</hd> <p></p> <hd id="AN0182958062-32">Ethics Approval</hd> <p>This study was performed in line with the principles of the Declaration of Helsinki and was approved by the Institutional Review Board at Boise State University (protocol number 101-SB17-207).</p> <hd id="AN0182958062-33">Consent to Participate</hd> <p>All participants provided written informed consent to be in the study.</p> <hd id="AN0182958062-34">Competing Interest</hd> <p>The authors declare no competing interests.</p> <hd id="AN0182958062-35">Supplementary Information</hd> <p>Below is the link to the electronic supplementary material.</p> <p>Graph: Supplementary file1 (DOCX 25 KB)</p> <hd id="AN0182958062-36">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0182958062-37"> <title> References </title> <blist> <bibl id="bib1" idref="ref42" type="bt">1</bibl> <bibtext> Albers B, Metz A, Burke K. Implementation support practitioners—A proposal for consolidating a diverse evidence base. BMC Health Services Research. 2020; 20; 1: 368. 10.1186/s12913-020-05145-1. 32357877. 7193379</bibtext> </blist> <blist> <bibl id="bib2" idref="ref43" type="bt">2</bibl> <bibtext> Algozzine, B, Barrett, S, Eber, L, Horner, R, Lewis, T, Putnam, B, Swain-Bradway, J, McIntosh, K, & Sugai, G. (2019). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. <ulink href="http://www.pbis.org">http://www.pbis.org</ulink>.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref32" type="bt">3</bibl> <bibtext> Baffsky R, Ivers R, Cullen P, Wang J, McGillivray L, Torok M. Strategies for enhancing the implementation of universal mental health prevention programs in schools: A systematic review. Prevention Science. 2023; 24; 2: 337-352. 10.1007/s11121-022-01434-9. 36098892</bibtext> </blist> <blist> <bibl id="bib4" idref="ref33" type="bt">4</bibl> <bibtext> Baffsky R, Ivers R, Cullen P, McGillivray L, Werner-Seidler A, Calear AL, Batterham PJ, Toumbourou JW, Stokes R, Kotselas P, Prendergast T, Torok M. Co-design and development of implementation strategies: Enhancing the PAX Good Behaviour Game in Australian Schools. Journal of Prevention. 2023; 44; 6: 679-704. 10.1007/s10935-023-00749-9. 37741909</bibtext> </blist> <blist> <bibl id="bib5" idref="ref34" type="bt">5</bibl> <bibtext> Becker KD, Bohnenkamp J, Domitrovich C, Keperling JP, Ialongo NS. Online training for teachers delivering evidence-based preventive interventions. School Mental Health. 2014; 6: 225-236. 10.1007/s12310-014-9124-x</bibtext> </blist> <blist> <bibl id="bib6" idref="ref2" type="bt">6</bibl> <bibtext> Blackmon BJ, Robison SB, Rhodes JLF. Examining the influence of risk factors across rural and urban communities. Journal of the Society for Social Work and Research. 2016; 7; 4: 615-638. 10.1086/689355</bibtext> </blist> <blist> <bibl id="bib7" idref="ref6" type="bt">7</bibl> <bibtext> Bradshaw, C. P. (2023). Leveraging implementation science to improve the scale-up of school mental health programming. In S.W. Evans, J. S. Owens, C. P. Bradshaw, & M. D. Weist (Eds.), Handbook of school mental health: Innovations in science and practice (pp. 409–417). Springer. https://doi.org/10.1007/978-3-031-20006-9</bibtext> </blist> <blist> <bibl id="bib8" idref="ref19" type="bt">8</bibl> <bibtext> Bradshaw CP, Koth CW, Bevans KB, Ialongo N, Leaf PJ. The impact of school-wide positive behavioral interventions and supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly. 2008; 23; 4: 462-473. 10.1037/a0012883</bibtext> </blist> <blist> <bibl id="bib9" idref="ref20" type="bt">9</bibl> <bibtext> Bradshaw CP, Koth CW, Thornton LA, Leaf PJ. Altering school climate through school-wide positive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science. 2009; 10; 2: 100. 10.1007/s11121-008-0114-9. 19011963</bibtext> </blist> <blist> <bibtext> Bradshaw CP, Mitchell MM, Leaf PJ. Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions. 2010; 12; 3: 133-148. 10.1177/1098300709334798</bibtext> </blist> <blist> <bibtext> Bradshaw CP, Waasdorp TE, Leaf PJ. Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics. 2012; 130; 5: e1136-e1145. 10.1542/peds.2012-0243. 23071207. 3483890</bibtext> </blist> <blist> <bibtext> Calvert HG, McQuilkin M, Havlicak A, Lewis T, Turner L. Acceptability, appropriateness, and feasibility of rural school support strategies for behavioral interventions: A mixed methods evaluation over two years of a hybrid type 3 implementation-effectiveness trial. Implementation Science Communications. 2023; 4; 1: 92. 10.1186/s43058-023-00478-4. 37568206. 10416374</bibtext> </blist> <blist> <bibtext> Chaparro EA, Kittelman A, McDaniel SC, Peshak George H, VanLone J, So B, SoLing. Examining rural school implementation of positive behavioral supports across tiers. Rural Special Education Quarterly. 2022; 41; 3: 116-128. 10.1177/87568705221097114</bibtext> </blist> <blist> <bibtext> Chilenski SM, Welsh JA, Perkins DF, Hoffman L. Going the distance: Technical assistance to community prevention coalitions and its impact on prevention coalitions over time. Consulting Psychology Journal. 2021; 73; 2: 181-198. 10.1037/cpb0000180. 34366643</bibtext> </blist> <blist> <bibtext> Chinman M, Hannah G, Wandersman A, Ebener P, Hunter SB, Imm P, Sheldon J. Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology. 2005; 35; 3–4: 143-157. 10.1007/s10464-005-3390-6. 15909791</bibtext> </blist> <blist> <bibtext> Connors EH, Smith-Millman M, Bohnenkamp JH, Carter T, Lever N, Hoover SA. Can we move the needle on school mental health quality through systematic quality improvement collaboratives?. School Mental Health. 2020; 12; 3: 478-492. 10.1007/s12310-020-09374-x. 34322180. 8315192</bibtext> </blist> <blist> <bibtext> Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science. 2019; 20; 6: 914-935. 10.1007/s11121-019-01017-1. 31152328. 8943907</bibtext> </blist> <blist> <bibtext> Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care. 2012; 50; 3: 217. 10.1097/MLR.0b013e3182408812. 22310560. 3731143</bibtext> </blist> <blist> <bibtext> Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: Are implementation effects out of control?. Clinical Psychology Review. 1998; 18; 1: 23-45. 10.1016/S0272-7358(97)00043-3. 9455622. 1:STN:280:DyaK1c7hs1yjsg%3D%3D</bibtext> </blist> <blist> <bibtext> Domitrovich CE, Bradshaw CP, Poduska JM, Hoagwood K, Buckley JA, Olin S, Romanelli LH, Leaf PJ, Greenberg MT, Ialongo NS. Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion. 2008; 1; 3: 6-28. 10.1080/1754730X.2008.9715730. 27182282. 4865398</bibtext> </blist> <blist> <bibtext> Domitrovich CE, Pas ET, Bradshaw CP, Becker KD, Keperling JP, Embry DD, Ialongo N. Individual and school organizational factors that influence implementation of the PAX Good Behavior Game intervention. Prevention Science. 2015; 16; 8: 1064-1074. 10.1007/s11121-015-0557-8. 25946968</bibtext> </blist> <blist> <bibtext> Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008; 41; 3: 327. 10.1007/s10464-008-9165-0. 18322790</bibtext> </blist> <blist> <bibtext> Fagan AA, Bumbarger BK, Barth RP, Bradshaw CP, Cooper BR, Supplee LH, Walker DK. Scaling up evidence-based interventions in US public systems to prevent behavioral health problems: Challenges and opportunities. Prevention Science. 2019; 20; 8: 1147-1168. 10.1007/s11121-019-01048-8. 31444621. 6881430</bibtext> </blist> <blist> <bibtext> Figas, K, Giannouchos, T, & Crouch, E. (2022). Rural-urban differences in child and adolescent mental health prior to and during the COVID-19 pandemic: Results from the National Survey of Children's Health. https://sc.edu/study/colleges_schools/public_health/research/research_centers/sc_rural_health_research_center/documents/youthmhbriefnov2022final.pdf</bibtext> </blist> <blist> <bibtext> Fixsen, D. L, Naoom, S. F, Blase, K. A, Friedman, R. M, & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.</bibtext> </blist> <blist> <bibtext> Gottfredson DC, Cook TD, Gardner FEM, Gorman-Smith D, Howe GW, Sandler IN, Zafft KM. Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science. 2015; 16; 7: 893-926. 10.1007/s11121-015-0555-x. 25846268. 4579256</bibtext> </blist> <blist> <bibtext> Grasley-Boy NM, Gage NA, Lombardo M, Anderson L, Rila A. School-wide positive behavior interventions and supports in rural and urban California schools: Effects on fidelity of implementation and suspension outcomes. Rural Special Education Quarterly. 2022; 41; 2: 84-94. 10.1177/87568705221092766</bibtext> </blist> <blist> <bibtext> Grasley-Boy NM, Gage NA, Lombardo M, Anderson L. Re-examining the reliability and validity of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions. 2023; 25; 4: 215-226. 10.1177/10983007231167606</bibtext> </blist> <blist> <bibtext> Hoagwood, K, & Johnson, J. (2003). School psychology: A public health framework: I. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41(1), 3–21. https://doi.org/10.1016/S0022-4405(02)00141-3</bibtext> </blist> <blist> <bibtext> Hoover, S. A, & Mayworm, A. M. (2017). The benefits of school mental health. In K. D. Michael & J. P. Jameson (Eds.), Handbook of rural school mental health (pp. 3–16). Springer International Publishing. https://doi.org/10.1007/978-3-319-64735-7_1</bibtext> </blist> <blist> <bibtext> Horner RH, Sugai G, Smolkowski K, Eber L, Nakasato J, Todd AW, Esperanza J. A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions. 2009; 11; 3: 133-144. 10.1177/1098300709332067</bibtext> </blist> <blist> <bibtext> Katz J, Wandersman A. Technical assistance to enhance prevention capacity: A research synthesis of the evidence base. Prevention Science. 2016; 17: 417-428. 10.1007/s11121-016-0636-5. 26858179. 4839040</bibtext> </blist> <blist> <bibtext> Lambert D, Gale JA, Hartley D. Substance abuse by youth and young adults in rural America. The Journal of Rural Health. 2008; 24; 3: 221-228. 10.1111/j.1748-0361.2008.00162.x. 18643798</bibtext> </blist> <blist> <bibtext> Le LT, Anthony BJ, Bronheim SM, Holland CM, Perry DF. A technical assistance model for guiding service and systems change. The Journal of Behavioral Health Services & Research. 2016; 43; 3: 380-395. 10.1007/s11414-014-9439-2</bibtext> </blist> <blist> <bibtext> Lee A, Gage NA. Updating and expanding systematic reviews and meta-analyses on the effects of school-wide positive behavior interventions and supports. Psychology in the Schools. 2020; 57; 5: 783-804. 10.1002/pits.22336</bibtext> </blist> <blist> <bibtext> Lyon AR, Bruns EJ. From evidence to impact: Joining our best school mental health practices with our best implementation strategies. School Mental Health. 2019; 11; 1: 106-114. 10.1007/s12310-018-09306-w. 31709018. 6839825</bibtext> </blist> <blist> <bibtext> McDaniel SC, Bloomfield BS. School-wide positive behavior support telecoaching in a rural district. Journal of Educational Technology Systems. 2020; 48; 3: 335-355. 10.1177/0047239519886283</bibtext> </blist> <blist> <bibtext> McDaniel, S. C, & Bloomfield, B. S. (2020b). PBIS in rural America: Addressing barriers and building on strengths. Eugene, OR: Center on PBIS, University of Oregon. https://assets-global.website-files.com/5d3725188825e071f1670246/5eaca3b0827083011f4168a9_PBIS%20in%20Rural%20America%202019%20RDQ%20Brief.pdf</bibtext> </blist> <blist> <bibtext> McIntosh K, Massar MM, Algozzine RF, George HP, Horner RH, Lewis TJ, Swain-Bradway J. Technical adequacy of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions. 2017; 19; 1: 3-13. 10.1177/1098300716637193</bibtext> </blist> <blist> <bibtext> Mercer SH, McIntosh K, Hoselton R. Comparability of fidelity measures for assessing tier 1 school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions. 2017; 19; 4: 195-204. 10.1177/1098300717693384</bibtext> </blist> <blist> <bibtext> Metz A, Albers B, Burke K, Bartley L, Louison L, Ward C, Farley A. Implementation practice in human service systems: Understanding the principles and competencies of professionals who support implementation. Human Service Organizations: Management, Leadership & Governance. 2021; 45; 3: 238-259. 10.1080/23303131.2021.1895401</bibtext> </blist> <blist> <bibtext> Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology. 2012; 50: 462-480. 10.1007/s10464-012-9522-x. 22644083</bibtext> </blist> <blist> <bibtext> Molloy LE, Moore JE, Trail J, Van Epps JJ, Hopfer S. Understanding real-world implementation quality and "active ingredients" of PBIS. Prevention Science. 2013; 14; 6: 593-605. 10.1007/s11121-012-0343-9. 23408283. 4032114</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics. (2022, August 10). The status of rural education. https://nces.ed.gov/programs/coe/pdf/coe_tla.pdf</bibtext> </blist> <blist> <bibtext> National Center for Education Statistics. (2021). Common core of data America's public schools. NCES. https://nces.ed.gov/ccd/ccddata.asp</bibtext> </blist> <blist> <bibtext> Noltemeyer, A, Palmer, K, James, A. G, & Petrasek, M. (2019). Disciplinary and achievement outcomes associated with school-wide positive behavioral interventions and supports implementation level. School Psychology Review, 48(1), 81–87. https://doi.org/10.17105/SPR-2017-0131.V48-1</bibtext> </blist> <blist> <bibtext> Olson JR, Coldiron JS, Parigoris RM, Zabel MD, Matarese M, Bruns EJ. Developing an evidence-based technical assistance model: A process evaluation of the National Training and Technical Assistance Center for child, youth, and family mental health. The Journal of Behavioral Health Services & Research. 2020; 47; 3: 312-330. 10.1007/s11414-020-09686-5</bibtext> </blist> <blist> <bibtext> Orenstein, S, Connors, E, Fields, P, Cushing, K, Yarnell, J, Bohnenkamp, J, Hoover, S, & Lever, N. (2023). Advancing school mental health quality through national learning communities. In S. W. Evans, J. S. Owens, C. P. Bradshaw, & M. D. Weist (Eds.), Handbook of School Mental Health: Innovations in Science and Practice (pp. 215–231). Springer. https://doi.org/10.1007/978-3-031-20006-9_15</bibtext> </blist> <blist> <bibtext> Orkin, A. M, Gill, P. J, Ghersi, D, Campbell, L, Sugarman, J, Emsley, R, Steg, P. G, Weijer, C, Simes, J, Rombey, T, Williams, H. C, Wittes, J, Moher, D, Richards, D. P, Kasamon, Y, Getz, K, Hopewell, S, Dickersin, K, Wu, T, ... CONSERVE Group. (2021). Guidelines for reporting trial protocols and completed trials modified due to the COVID-19 pandemic and other extenuating circumstances: The CONSERVE 2021 Statement. JAMA, 326(3), 257–265. https://doi.org/10.1001/jama.2021.9941</bibtext> </blist> <blist> <bibtext> Pas ET, Kaiser L, Owens JSEvans SW, Owens JS, Bradshaw CP, Weist MD. Innovative approaches to coaching teachers in implementing Tier 1 and Tier 2 classroom interventions. Handbook of School Mental Health: Innovations in Science and Practice. 2023; Springer: 343-355. 10.1007/978-3-031-20006-9_23</bibtext> </blist> <blist> <bibtext> Pas, E. T, Ryoo, J. H, Musci, R. J, & Bradshaw, C. P. (2019). A state-wide quasi-experimental effectiveness study of the scale-up of school-wide positive behavioral interventions and supports. Journal of School Psychology, 73, 41–55. https://doi.org/10.1016/j.jsp.2019.03.001</bibtext> </blist> <blist> <bibtext> Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project. Implementation Science. 2015; 10; 1: 21. 10.1186/s13012-015-0209-1. 25889199. 4328074</bibtext> </blist> <blist> <bibtext> Proctor EK, Powell BJ, McMillen JC. Implementation strategies: Recommendations for specifying and reporting. Implementation Science. 2013; 8; 1: 139. 10.1186/1748-5908-8-139. 24289295. 3882890</bibtext> </blist> <blist> <bibtext> Sugai G, Horner RR. A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review. 2006; 35; 2: 245-259. 10.1080/02796015.2006.12087989</bibtext> </blist> <blist> <bibtext> Sugai G, Lewis-Palmer T, Todd A, Horner RH. School-wide evaluation tool. 2001; University of Oregon</bibtext> </blist> <blist> <bibtext> Turner L, Calvert HG, Fleming CM, Lewis T, Siebert C, Anderson N, Castleton T, Havlicak A, McQuilkin M. Study protocol for a cluster-randomized trial of a bundle of implementation support strategies to improve the fidelity of implementation of schoolwide positive behavioral interventions and supports in rural schools. Contemporary Clinical Trials Communications. 2022; 28: 100949. 10.1016/j.conctc.2022.100949. 35782635. 9240699</bibtext> </blist> <blist> <bibtext> von der Embse N, Rutherford L, Mankin A, Jenkins A. Demonstration of a trauma-informed assessment to intervention model in a large urban school district. School Mental Health. 2019; 11; 2: 276-289. 10.1007/s12310-018-9294-z</bibtext> </blist> <blist> <bibtext> Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology. 2012; 50; 3–4: 445-459. 10.1007/s10464-012-9509-7. 22538406</bibtext> </blist> <blist> <bibtext> Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J. Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology. 2008; 41; 3: 171-181. 10.1007/s10464-008-9174-z. 18302018</bibtext> </blist> <blist> <bibtext> Williams NJ, Preacher KJ, Allison PD, Mandell DS, Marcus SC. Required sample size to detect mediation in 3-level implementation studies. Implementation Science. 2022; 17; 1: 66. 10.1186/s13012-022-01235-2. 36183090. 9526963</bibtext> </blist> <blist> <bibtext> Zubkoff L, Neily J, Mills PD. How to do a virtual breakthrough series collaborative. Journal of Medical Systems. 2019; 43: 27. 10.1007/s10916-018-1126-z. 30603939</bibtext> </blist> </ref> <aug> <p>By Hannah G. Calvert; Christopher M. Fleming; Michaela Lowe; Teri Lewis; Carl F. Siebert; Ashley Havlicak; Nate Anderson; Tate Castleton and Lindsey Turner</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib33" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib24" firstref="ref3"></nolink> <nolink nlid="nl3" bibid="bib30" firstref="ref4"></nolink> <nolink nlid="nl4" bibid="bib23" firstref="ref5"></nolink> <nolink nlid="nl5" bibid="bib36" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib22" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib58" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib20" firstref="ref10"></nolink> <nolink nlid="nl9" bibid="bib50" firstref="ref11"></nolink> <nolink nlid="nl10" bibid="bib15" firstref="ref12"></nolink> <nolink nlid="nl11" bibid="bib32" firstref="ref14"></nolink> <nolink nlid="nl12" bibid="bib34" firstref="ref15"></nolink> <nolink nlid="nl13" bibid="bib48" firstref="ref16"></nolink> <nolink nlid="nl14" bibid="bib61" firstref="ref17"></nolink> <nolink nlid="nl15" bibid="bib54" firstref="ref18"></nolink> <nolink nlid="nl16" bibid="bib11" firstref="ref21"></nolink> <nolink nlid="nl17" bibid="bib31" firstref="ref22"></nolink> <nolink nlid="nl18" bibid="bib46" firstref="ref23"></nolink> <nolink nlid="nl19" bibid="bib51" firstref="ref24"></nolink> <nolink nlid="nl20" bibid="bib29" firstref="ref26"></nolink> <nolink nlid="nl21" bibid="bib13" firstref="ref28"></nolink> <nolink nlid="nl22" bibid="bib44" firstref="ref29"></nolink> <nolink nlid="nl23" bibid="bib43" firstref="ref30"></nolink> <nolink nlid="nl24" bibid="bib38" firstref="ref31"></nolink> <nolink nlid="nl25" bibid="bib10" firstref="ref36"></nolink> <nolink nlid="nl26" bibid="bib37" firstref="ref37"></nolink> <nolink nlid="nl27" bibid="bib57" firstref="ref38"></nolink> <nolink nlid="nl28" bibid="bib17" firstref="ref39"></nolink> <nolink nlid="nl29" bibid="bib21" firstref="ref40"></nolink> <nolink nlid="nl30" bibid="bib56" firstref="ref41"></nolink> <nolink nlid="nl31" bibid="bib53" firstref="ref46"></nolink> <nolink nlid="nl32" bibid="bib18" firstref="ref47"></nolink> <nolink nlid="nl33" bibid="bib49" firstref="ref49"></nolink> <nolink nlid="nl34" bibid="bib59" firstref="ref50"></nolink> <nolink nlid="nl35" bibid="bib42" firstref="ref54"></nolink> <nolink nlid="nl36" bibid="bib41" firstref="ref60"></nolink> <nolink nlid="nl37" bibid="bib45" firstref="ref62"></nolink> <nolink nlid="nl38" bibid="bib39" firstref="ref64"></nolink> <nolink nlid="nl39" bibid="bib40" firstref="ref65"></nolink> <nolink nlid="nl40" bibid="bib35" firstref="ref66"></nolink> <nolink nlid="nl41" bibid="bib28" firstref="ref67"></nolink> <nolink nlid="nl42" bibid="bib27" firstref="ref69"></nolink> <nolink nlid="nl43" bibid="bib25" firstref="ref80"></nolink> <nolink nlid="nl44" bibid="bib47" firstref="ref82"></nolink> <nolink nlid="nl45" bibid="bib14" firstref="ref83"></nolink> <nolink nlid="nl46" bibid="bib12" firstref="ref84"></nolink> <nolink nlid="nl47" bibid="bib16" firstref="ref86"></nolink> <nolink nlid="nl48" bibid="bib52" firstref="ref90"></nolink> <nolink nlid="nl49" bibid="bib26" firstref="ref94"></nolink> <nolink nlid="nl50" bibid="bib55" firstref="ref95"></nolink> <nolink nlid="nl51" bibid="bib60" firstref="ref96"></nolink> <nolink nlid="nl52" bibid="bib19" firstref="ref97"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1460850
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Training and Technical Assistance Increase the Fidelity of Implementation of a Universal Prevention Initiative in Rural Schools: Results from a 3-Year Cluster-Randomized Trial
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Hannah+G%2E+Calvert%22">Hannah G. Calvert</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-6067-693X">0000-0002-6067-693X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Christopher+M%2E+Fleming%22">Christopher M. Fleming</searchLink><br /><searchLink fieldCode="AR" term="%22Michaela+Lowe%22">Michaela Lowe</searchLink><br /><searchLink fieldCode="AR" term="%22Teri+Lewis%22">Teri Lewis</searchLink><br /><searchLink fieldCode="AR" term="%22Carl+F%2E+Siebert%22">Carl F. Siebert</searchLink><br /><searchLink fieldCode="AR" term="%22Ashley+Havlicak%22">Ashley Havlicak</searchLink><br /><searchLink fieldCode="AR" term="%22Nate+Anderson%22">Nate Anderson</searchLink><br /><searchLink fieldCode="AR" term="%22Tate+Castleton%22">Tate Castleton</searchLink><br /><searchLink fieldCode="AR" term="%22Lindsey+Turner%22">Lindsey Turner</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Prevention+Science%22"><i>Prevention Science</i></searchLink>. 2025 26(1):56-68.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 13
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Institute of Justice (NIJ) (DOJ)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: 2017CKBX0021
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Elementary+School+Students%22">Elementary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Positive+Behavior+Supports%22">Positive Behavior Supports</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Behavior%22">Student Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Rural+Schools%22">Rural Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Evidence+Based+Practice%22">Evidence Based Practice</searchLink><br /><searchLink fieldCode="DE" term="%22Leadership+Training%22">Leadership Training</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+Development%22">Professional Development</searchLink><br /><searchLink fieldCode="DE" term="%22Resource+Allocation%22">Resource Allocation</searchLink><br /><searchLink fieldCode="DE" term="%22Technical+Assistance%22">Technical Assistance</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+Learning%22">Electronic Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Workshops%22">Workshops</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Gains%22">Achievement Gains</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Environment%22">Educational Environment</searchLink><br /><searchLink fieldCode="DE" term="%22Summer+Programs%22">Summer Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Fidelity%22">Fidelity</searchLink><br /><searchLink fieldCode="DE" term="%22Prevention%22">Prevention</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s11121-025-01776-0
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1389-4986<br />1573-6695
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The need for well-implemented evidence-based interventions (EBIs) for the prevention of behavioral issues among children and adolescents is substantial. In rural areas, the need often matches or surpasses that of urban areas. Schools have a wide reach for prevention-focused EBIs. However, implementation in rural schools is often hindered by limited resources and capacity. Rural School Support Strategies (RS3) are a bundle of implementation supports that address implementation challenges in rural settings. They include providing additional leadership and coaching training, individualized technical assistance (mostly virtual), and monthly meetings of a virtual learning collaborative. A cluster-randomized Hybrid Type 3 implementation-effectiveness trial tested RS3 for implementing school-wide positive behavioral interventions and supports (PBIS), a universal prevention approach to improving student behavior, academic outcomes, and school climate. Forty rural schools received a multi-day training on PBIS each summer for 3 years. Half were randomized to also receive RS3 support. Linear and logistic regression models examined the effect of treatment condition and dosage of support on implementation fidelity for Tier 1 (universal) PBIS. Condition and dosage (number of hours) of support increased the odds of schools achieving the 70% threshold for adequate implementation fidelity. In the first year, the higher dosage of technical assistance events increased the likelihood of schools reaching fidelity, whereas later in the trial, the higher dosage of attendance at the virtual learning collaborative sessions yielded significant improvements in fidelity. Implications for accelerating the implementation of universal prevention initiatives in schools--particularly in rural settings--are discussed. This study was prospectively registered on ClinicalTrials.gov (NCT03736395), on November 9, 2018.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1460850
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1460850
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s11121-025-01776-0
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 56
    Subjects:
      – SubjectFull: Elementary School Students
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Positive Behavior Supports
        Type: general
      – SubjectFull: Student Behavior
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Rural Schools
        Type: general
      – SubjectFull: Evidence Based Practice
        Type: general
      – SubjectFull: Leadership Training
        Type: general
      – SubjectFull: Professional Development
        Type: general
      – SubjectFull: Resource Allocation
        Type: general
      – SubjectFull: Technical Assistance
        Type: general
      – SubjectFull: Electronic Learning
        Type: general
      – SubjectFull: Workshops
        Type: general
      – SubjectFull: Achievement Gains
        Type: general
      – SubjectFull: Educational Environment
        Type: general
      – SubjectFull: Summer Programs
        Type: general
      – SubjectFull: Fidelity
        Type: general
      – SubjectFull: Prevention
        Type: general
    Titles:
      – TitleFull: Training and Technical Assistance Increase the Fidelity of Implementation of a Universal Prevention Initiative in Rural Schools: Results from a 3-Year Cluster-Randomized Trial
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Hannah G. Calvert
      – PersonEntity:
          Name:
            NameFull: Christopher M. Fleming
      – PersonEntity:
          Name:
            NameFull: Michaela Lowe
      – PersonEntity:
          Name:
            NameFull: Teri Lewis
      – PersonEntity:
          Name:
            NameFull: Carl F. Siebert
      – PersonEntity:
          Name:
            NameFull: Ashley Havlicak
      – PersonEntity:
          Name:
            NameFull: Nate Anderson
      – PersonEntity:
          Name:
            NameFull: Tate Castleton
      – PersonEntity:
          Name:
            NameFull: Lindsey Turner
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1389-4986
            – Type: issn-electronic
              Value: 1573-6695
          Numbering:
            – Type: volume
              Value: 26
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Prevention Science
              Type: main
ResultId 1