View in EDS HTML Full Text PDF Full Text

Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks across Child Mental Health Service Settings

Saved in:
Bibliographic Details
Title: Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks across Child Mental Health Service Settings
Language: English
Authors: Stephanie A. Moore (ORCID 0000-0003-1550-8344), Jennifer McGrory Cooper (ORCID 0000-0003-0509-1653), JoAnne Malloy (ORCID 0000-0002-7403-476X), Aaron R. Lyon (ORCID 0000-0003-3657-5060)
Source: Administration and Policy in Mental Health and Mental Health Services Research. 2024 51(2):172-195.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 24
Publication Date: 2024
Sponsoring Agency: National Institute of Mental Health (NIMH) (DHHS/NIH)
Contract Number: T32MH109436
R01MH119148
Document Type: Journal Articles
Reports - Research
Descriptors: Mental Health, Health Services, Context Effect, Intervention, Positive Behavior Supports, Primary Health Care, Community Health Services, Componential Analysis
DOI: 10.1007/s10488-023-01320-8
ISSN: 0894-587X
1573-3289
Abstract: Multilevel service delivery frameworks are approaches to structuring and organizing a spectrum of evidence-based services and supports, focused on assessment, prevention, and intervention designed for the local context. Exemplar frameworks in child mental health include positive behavioral interventions and supports in education, collaborative care in primary care, and systems of care in community mental health settings. Yet, their high-quality implementation has lagged. This work proposes a conceptual foundation for multilevel service delivery frameworks spanning diverse mental health service settings that can inform development of strategic implementation supports. We draw upon the existing literature for three exemplar multilevel service delivery frameworks in different child mental health service settings to (1) identify core components common to each framework, and (2) to highlight prominent implementation determinants that interface with each core component. Six interrelated components of multilevel service delivery frameworks were identified, including, (1) a systems-level approach, (2) data-driven problem solving and decision-making, (3) multiple levels of service intensity using evidence-based practices, (4) cross-linking service sectors, (5) multiple providers working together, including in teams, and (6) built-in implementation strategies that facilitate delivery of the overall model. Implementation determinants that interface with core components were identified at each contextual level. The conceptual foundation provided in this paper has the potential to facilitate cross-sector knowledge sharing, promote generalization across service settings, and provide direction for researchers, system leaders, and implementation intermediaries/practitioners working to strategically support the high-quality implementation of these frameworks.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1434362
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF1JJoA0rGVxT78fD3xuDaMAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDIbW-KwJBvP2scAVyQIBEICBmiHy9tb7uWiM2XVC8g6ozuuvxnnyrd_K7FcrHC2HSBuNiUa0aWYCz7H1a1EP3Nyw9BxmMN_PdHU5NlkgeOJiw6ZkKD8IKnVmiUiIVcXcXKvuHDSkXUD09NwbBnYmvu5VKCgxHUdY5IWAyumWcdKZWF2HSLQKn1XthNnvG-yJUBP29F1QvI484ish3T-U87DMPvxZRsFZDplpg_8=
Text:
  Availability: 1
  Value: <anid>AN0175304773;[1xdo]01mar.24;2024Feb09.05:50;v2.2.500</anid> <title id="AN0175304773-1">Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks Across Child Mental Health Service Settings </title> <p>Multilevel service delivery frameworks are approaches to structuring and organizing a spectrum of evidence-based services and supports, focused on assessment, prevention, and intervention designed for the local context. Exemplar frameworks in child mental health include positive behavioral interventions and supports in education, collaborative care in primary care, and systems of care in community mental health settings. Yet, their high-quality implementation has lagged. This work proposes a conceptual foundation for multilevel service delivery frameworks spanning diverse mental health service settings that can inform development of strategic implementation supports. We draw upon the existing literature for three exemplar multilevel service delivery frameworks in different child mental health service settings to (<reflink idref="bib1" id="ref1">1</reflink>) identify core components common to each framework, and (<reflink idref="bib2" id="ref2">2</reflink>) to highlight prominent implementation determinants that interface with each core component. Six interrelated components of multilevel service delivery frameworks were identified, including, (<reflink idref="bib1" id="ref3">1</reflink>) a systems-level approach, (<reflink idref="bib2" id="ref4">2</reflink>) data-driven problem solving and decision-making, (<reflink idref="bib3" id="ref5">3</reflink>) multiple levels of service intensity using evidence-based practices, (<reflink idref="bib4" id="ref6">4</reflink>) cross-linking service sectors, (<reflink idref="bib5" id="ref7">5</reflink>) multiple providers working together, including in teams, and (<reflink idref="bib6" id="ref8">6</reflink>) built-in implementation strategies that facilitate delivery of the overall model. Implementation determinants that interface with core components were identified at each contextual level. The conceptual foundation provided in this paper has the potential to facilitate cross-sector knowledge sharing, promote generalization across service settings, and provide direction for researchers, system leaders, and implementation intermediaries/practitioners working to strategically support the high-quality implementation of these frameworks.</p> <p>Keywords: Child mental health; Core components; Implementation determinants; Service delivery frameworks</p> <hd id="AN0175304773-2">Introduction</hd> <p>Despite rising rates of youth mental health disorder and deaths by suicide (Houtrow et al., [<reflink idref="bib55" id="ref9">55</reflink>]; Miron et al., [<reflink idref="bib81" id="ref10">81</reflink>]), few children and adolescents receive needed mental health treatment (Costello et al., [<reflink idref="bib31" id="ref11">31</reflink>]; Merikangas et al., [<reflink idref="bib79" id="ref12">79</reflink>]; Simon et al., [<reflink idref="bib109" id="ref13">109</reflink>]). Discrepancies in receipt of high-quality care are further pronounced for youth of color and families living in communities experiencing high rates of poverty (Alegría et al., [<reflink idref="bib5" id="ref14">5</reflink>]; Hodgkinson et al., [<reflink idref="bib51" id="ref15">51</reflink>]; Locke et al., [<reflink idref="bib70" id="ref16">70</reflink>]). When left untreated, mental health disorders that arise in childhood and adolescence persist and contribute to later deleterious outcomes (Office of the Surgeon General, [<reflink idref="bib87" id="ref17">87</reflink>]). Accordingly, increasing youths' access to high-quality prevention, early intervention, and mental health treatment services is critical.</p> <hd id="AN0175304773-3">Multilevel Service Delivery Frameworks</hd> <p>Schools, outpatient settings, and primary care are the most common loci of mental health care for youth who receive services (Duong et al., [<reflink idref="bib37" id="ref18">37</reflink>]). Across these principal youth mental health service sectors, service delivery frameworks designed to address the full range of youth mental health needs (i.e., via prevention, early intervention, and treatment) are championed as best practices for increasing access to care and improving outcomes—exemplars include positive behavioral interventions and supports (PBIS) in education settings (e.g., Bradshaw et al., [<reflink idref="bib15" id="ref19">15</reflink>]; Eiraldi et al., [<reflink idref="bib38" id="ref20">38</reflink>]), the collaborative care model (CoCM) in primary care settings (e.g., Goodrich et al., [<reflink idref="bib47" id="ref21">47</reflink>]; Richardson et al., [<reflink idref="bib97" id="ref22">97</reflink>]), and system of care (SoC) in community mental health settings (e.g., Stroul et al., [<reflink idref="bib116" id="ref23">116</reflink>], [<reflink idref="bib114" id="ref24">114</reflink>]). As used here, these <emph>multilevel service delivery frameworks</emph> are defined as approaches to structuring and organizing a spectrum of research-based services and supports, focused on assessment, prevention, and intervention matched to the needs of the target population and appropriate for the service context. Although the application of these frameworks is common across many youth (and adult) service sectors, integration of their disparate literatures to identify common components or generalizable findings about their implementation is needed.</p> <p>The features and services that comprise multilevel service delivery frameworks are situated at multiple tiers—including practices of varied intensity targeted toward whole populations, small groups, or individuals (i.e., tiered service delivery or primary, secondary, and tertiary prevention). Each tier may also focus on multiple actors within that service system (e.g., clients, providers, administrative units, and policymaking/legislative bodies). We use the term "frameworks" to be inclusive of the varied components within these approaches to multilevel service delivery, including discrete intervention and assessment methods as well as the structures that facilitate their delivery. Integral to multilevel service delivery frameworks is the specification of how components relate to one another and are administratively managed. For example, services of varying levels of intensity that occur across multiple entities may be coordinated by teams who use data to guide decision-making. Although multilevel service delivery frameworks are not limited to youth mental health services (e.g., CoCM has been implemented far more extensively with adults than with youth), we focus on child-serving treatment settings for myriad reasons including the current youth mental health crisis (Office of the Surgeon General, [<reflink idref="bib87" id="ref25">87</reflink>]) and because there are several well-articulated examples of multilevel service delivery frameworks for the primary settings in which youth typically access mental health care.</p> <hd id="AN0175304773-4">Implementation of Multilevel Frameworks</hd> <p>Multilevel service delivery frameworks hold great potential for addressing a wide range of youth mental health needs, but this potential is unlikely to be achieved without widespread adoption and high-quality implementation within their respective service settings (Durlak & Dupree, [<reflink idref="bib36" id="ref26">36</reflink>]). Frameworks such as CoCM, PBIS, and SoC are increasingly adopted (AIMS Center, [<reflink idref="bib3" id="ref27">3</reflink>]; Center on PBIS, [<reflink idref="bib25" id="ref28">25</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref29">114</reflink>]), however, their implementation is fraught with challenges. For example, insufficient organizational capacity, piecemeal implementation of key features (i.e., intervention and assessment), and ineffective development and/or use of data systems (e.g., to inform intervention selection or monitor progress; e.g., Fabiano & Evans, [<reflink idref="bib39" id="ref30">39</reflink>]; Overbeck et al., [<reflink idref="bib88" id="ref31">88</reflink>]) can result in implementation outcomes that differ substantially from the intended model (e.g., Molloy et al., [<reflink idref="bib82" id="ref32">82</reflink>]). When considering multilevel service delivery frameworks, implementation outcomes such as "fidelity" might refer either to <emph>intervention</emph> fidelity—focused on the delivery of service-recipient-facing component practices (also called intervention integrity or treatment fidelity)—or <emph>implementation strategy</emph> fidelity—the extent to which implementation practices and techniques were delivered as intended. Given the complexity of these frameworks, both are important in understanding child outcomes (Saldana, [<reflink idref="bib101" id="ref33">101</reflink>]; Sanetti et al., [<reflink idref="bib105" id="ref34">105</reflink>]).</p> <p>Similar to more circumscribed evidence-based interventions (Durlak & Dupree, [<reflink idref="bib36" id="ref35">36</reflink>]; Lawson et al., [<reflink idref="bib68" id="ref36">68</reflink>]), identifying core components of multilevel service delivery frameworks can lay the groundwork for tracking and evaluating implementation of these frameworks. For example, identifying which components service systems are most (or least) often implementing with fidelity (intervention or strategy) allows us to better understand how a component's implementation impacts youth mental health and service outcomes. Whereas scholars have begun to explicate and investigate key features of individual multilevel service delivery frameworks, such as CoCM in primary care (Bao et al., [<reflink idref="bib11" id="ref37">11</reflink>]; Wood et al., [<reflink idref="bib132" id="ref38">132</reflink>]) and PBIS in schools (Molloy et al., [<reflink idref="bib82" id="ref39">82</reflink>]), this work is framework- and setting-specific, and thus limited in its generalizability to other frameworks and settings. Identifying multilevel service delivery frameworks as a common phenomenon across mental health service sectors is an important next step. Despite the pertinence of prior research for enhancing implementation of the targeted framework, research that spans service settings and frameworks may enable knowledge sharing across scholars and professionals within different service systems. In this paper, we argue that synthesizing the evidence relative to core features of varied multilevel service delivery frameworks has the potential to inform a larger understanding of those frameworks and, in doing so, improve their implementation and sustainment across settings.</p> <p>As implementation of service delivery frameworks has been found to vary by core component (e.g., Molloy et al., [<reflink idref="bib82" id="ref40">82</reflink>]), understanding how their core features interface with implementation barriers and facilitators (i.e., determinants) is also critical for supporting their high-quality implementation. Implementation strategies are likely to be most effective when strategies are tailored to the implementation determinants that are most salient for the target innovation and its context (Baker et al., [<reflink idref="bib10" id="ref41">10</reflink>]; Lau et al., [<reflink idref="bib67" id="ref42">67</reflink>]; Powell et al., [<reflink idref="bib92" id="ref43">92</reflink>]). Research in support of this goal is emerging for exemplar multilevel service delivery frameworks. For example, a systematic review of CoCM for depression conducted by Wood et al. ([<reflink idref="bib132" id="ref44">132</reflink>]) identified barriers and facilitators to the implementation of specific components of CoCM (e.g., barriers to multi-professional teamwork included organization culture and negative staff attitudes to change, whereas facilitators included staff champions and peer learning and support). Scholars have also identified and investigated core components (e.g., Molloy et al., [<reflink idref="bib82" id="ref45">82</reflink>]; Center on PBIS, [<reflink idref="bib24" id="ref46">24</reflink>]) and salient implementation determinants (e.g., Fox et al., [<reflink idref="bib43" id="ref47">43</reflink>]) for PBIS. However, to our knowledge, research has yet to link implementation determinants to core features of exemplar frameworks in school or community care settings nor to the core components shared by multilevel service delivery frameworks. Given the research in primary care settings that indicates important determinants vary by component (Wood et al., [<reflink idref="bib132" id="ref48">132</reflink>]), this study works to begin identifying determinants for core components of multilevel service delivery frameworks.</p> <hd id="AN0175304773-5">Purpose of this Paper and Rationale for Selected Frameworks</hd> <p>There are two primary aims for this paper. First, we draw from exemplar multilevel service delivery frameworks in child mental health service settings to propose a conceptual foundation and identify core components. Second, with a goal of identifying commonalities across frameworks and service settings, we highlight prominent implementation determinants for each core component. Our overarching goal for this project is to facilitate knowledge sharing between researchers, system leaders, and implementation intermediaries/practitioners who may focus most explicitly on one service system but would benefit from a broader knowledge base. Our preliminary analysis can be built upon in future research that identifies how implementation determinants affect implementation outcomes, which can ultimately inform identification and testing of tailored implementation strategies.</p> <p>In selecting the multi-level service delivery frameworks that served as the foundation for this paper, the author team developed inclusion criteria to assist in identifying a central and representative model used in each of the settings in which youth most commonly access mental health care—schools, outpatient settings, and primary care (Duong et al., [<reflink idref="bib37" id="ref49">37</reflink>]). Specifically, inclusion criteria prescribed that each model should: (a) address the full range of youth mental health needs through services provided at multiple levels of intensity (i.e., via prevention, early intervention, and treatment), (b) have empirical support, and (c) have research demonstrating the widespread adoption of the model within the given service sector (e.g., based on the number of agencies implementing, support of framework in state or federal policies, etc.). These criteria were developed organically at the outset of this paper's conceptualization and the review process guided by expert consensus and literature focused on each framework. For criterion A, the author team discussed all possible multi-level frameworks in the three mental health service settings. In some cases, one framework emerged as the best exemplar for that particular setting (i.e., CoCM and SoC) based on consensus opinion, empirical research demonstrating efficacy (e.g., Asarnow et al., [<reflink idref="bib8" id="ref50">8</reflink>]; Kolko et al., [<reflink idref="bib63" id="ref51">63</reflink>]), and widespread adoption of the framework (e.g., Graaf et al., [<reflink idref="bib48" id="ref52">48</reflink>]; Kutash et al., [<reflink idref="bib66" id="ref53">66</reflink>]). For other settings (i.e., schools), several multi-level frameworks were considered (e.g., Interconnected Systems Framework [ISF], Multitiered Systems of Support-Behavior [MTSS-B]) and discussed relative to the criteria before consensus was reached. For criteria B and C, the authors were guided by established definitions for evidence-based interventions (e.g., What Works Clearinghouse) and reference to widespread adoption of the framework as outlined in either federal and/or state policies or empirical studies (e.g., Asarnow et al., [<reflink idref="bib8" id="ref54">8</reflink>]; Bradshaw et al., [<reflink idref="bib17" id="ref55">17</reflink>]; Miller et al., [<reflink idref="bib77" id="ref56">77</reflink>]; Pas & Bradshaw, [<reflink idref="bib89" id="ref57">89</reflink>]; Solberg et al., [<reflink idref="bib111" id="ref58">111</reflink>]).</p> <p>Within the educational setting, positive behavioral interventions and supports (PBIS; Horner & Sugai, [<reflink idref="bib53" id="ref59">53</reflink>]) was selected given its recognition as one of the most widely adopted evidence-based frameworks in schools and districts. Adopted in over 25,000 schools (Center on PBIS, [<reflink idref="bib25" id="ref60">25</reflink>]), PBIS is a three-tiered prevention framework associated with reductions in students' problematic behaviors and improvements in prosocial behavior (e.g., Bradshaw et al., [<reflink idref="bib16" id="ref61">16</reflink>]; McIntosh et al., [<reflink idref="bib73" id="ref62">73</reflink>]). Next, the collaborative care model (CoCM), which is a service model for integrating mental and behavioral health services into primary care, was selected (Goodrich et al., [<reflink idref="bib47" id="ref63">47</reflink>]; Richardson et al., [<reflink idref="bib97" id="ref64">97</reflink>]). CoCM has been studied in over 80 randomized controlled trials (AIMS Center, [<reflink idref="bib4" id="ref65">4</reflink>]), and meta-analyses have shown that CoCM is effective in bringing about positive mental health outcomes (e.g., Asarnow et al., [<reflink idref="bib8" id="ref66">8</reflink>]). Within community mental health settings, system of care (SoC; Stroul et al., [<reflink idref="bib116" id="ref67">116</reflink>], [<reflink idref="bib114" id="ref68">114</reflink>])—a framework to support youth with significant mental health concerns (Stroul & Friedman, [<reflink idref="bib118" id="ref69">118</reflink>])—was selected on the basis of its support through state and federal policy (Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, and U.S. Department of Health and Human Services, [<reflink idref="bib23" id="ref70">23</reflink>]) and association with improvements in youth emotional and behavioral health and effective use of services (Graaf et al., [<reflink idref="bib48" id="ref71">48</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref72">114</reflink>]). In Table 1, we provide an overview of each of these frameworks, including service setting, a brief description, key contributors/collaborators, applications, and a summary of the evidence-base.</p> <p>Table 1 Exemplary multilevel service delivery frameworks in child mental health service settings</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Characteristic</p></th><th align="left" colspan="3"><p>Exemplar framework</p></th></tr><tr><th align="left"><p>Collaborative care (CoCM)</p></th><th align="left"><p>Positive behavioral interventions and supports (PBIS)</p></th><th align="left"><p>System of care (SoC)</p></th></tr></thead><tbody><tr><td align="left"><p>Setting</p></td><td align="left"><p>Primary care</p></td><td align="left"><p>Education</p></td><td align="left"><p>Community-based mental health</p></td></tr><tr><td align="left"><p>Description</p></td><td align="left"><p>CoCM is a service model for integrating mental and behavioral health services into primary care. A multidisciplinary team—most often including a general practitioner or physician, mental health specialist, and a designated care coordinator—work together to coordinate population-based, client-centered care (e.g., Katon et al., <xref ref-type="bibr" rid="bibr58">2010</xref>; Lyon et al., <xref ref-type="bibr" rid="bibr72">2016</xref>; Unützer et al., <xref ref-type="bibr" rid="bibr124">2013</xref>).</p></td><td align="left"><p>PBIS is a three-tiered framework used in education to improve and integrate the data, systems, and practices affecting student outcomes on a schoolwide basis. Specifically, PBIS is a population-based approach to prevention in schools that involves the delivery of evidence-based services along a continuum based on student need (Jimerson et al., <xref ref-type="bibr" rid="bibr57">2016</xref>; McIntosh and Goodman, <xref ref-type="bibr" rid="bibr74">2016</xref>).</p></td><td align="left"><p>SoC is a framework targeted to support youth with significant mental health concerns in community mental health settings (Stroul & Friedman; <xref ref-type="bibr" rid="bibr118">1986</xref>). The framework focuses on (1) a comprehensive array of services and supports (e.g., care coordination via wraparound; Walker et al., <xref ref-type="bibr" rid="bibr127">2008</xref>), (2) structures and processes to facilitate service provision, and (3) a philosophy of service delivery that is youth and caregiver- driven and prioritizes culturally/linguistically responsive care (Stroul et al., <xref ref-type="bibr" rid="bibr114">2021</xref>).</p></td></tr><tr><td align="left"><p>Key contributors and collaborators</p></td><td align="left"><p>General practitioner or primary care physician; mental health specialists, such as a psychiatrist, psychologist, psychiatric nurse, or social worker; care manager</p></td><td align="left"><p>Family members, students, educators (e.g., teachers or other direct implementers), administrators and board members (school, district, region and state), and community partners and agency representatives and partners</p></td><td align="left"><p>Youth with behavioral health needs and their families, community-based mental health providers, child protection, juvenile justice, school personnel, and natural supports (e.g., extended family, church, neighbors, mentors)</p></td></tr><tr><td align="left"><p>Applications</p></td><td align="left"><p>Although originally developed for and extensively used with adult populations, CoCM has been applied across a variety of youth mental health challenges, including behavior problems, attention deficit hyperactivity disorder (ADHD), and anxiety in children (e.g., Kolko et al., <xref ref-type="bibr" rid="bibr62">2010</xref>; Kolko et al., <xref ref-type="bibr" rid="bibr63">2014</xref>), as well as depression in adolescents (e.g., Asarnow et al., <xref ref-type="bibr" rid="bibr7">2005</xref>; Richardson et al., <xref ref-type="bibr" rid="bibr96">2014</xref>).</p></td><td align="left"><p>PBIS is typically applied on a schoolwide basis to deliver practices and intervention systems that create positive, predictable, equitable and safe learning environments and enhance students' academic and behavioral outcomes (Center on PBIS, <xref ref-type="bibr" rid="bibr24">2015</xref>; McIntosh & Goodman, <xref ref-type="bibr" rid="bibr74">2016</xref>). PBIS can be integrated with other multi-tiered practices and systems focused on academics, such as Response to Intervention (RtI; Ikeda et al., <xref ref-type="bibr" rid="bibr56">1996</xref>), and student mental health (e.g., Interconnected Systems Framework, ISF; Splett et al., <xref ref-type="bibr" rid="bibr112">2017</xref>).</p></td><td align="left"><p>SoC has been applied and interpreted across U.S. regions, populations, area of emphasis, and child-serving settings (e.g., mental health, child welfare; Stroul et al., <xref ref-type="bibr" rid="bibr117">2015</xref>). SoC is the foundation for the Federal children's mental health initiative (Stroul et al., <xref ref-type="bibr" rid="bibr115">2010</xref>), which continues to provide resources to expand and sustain SoC nationwide via the Substance Abuse and Mental Health Services Administration (SAMHSA, <xref ref-type="bibr" rid="bibr120">2020</xref>) Center for Mental Health Services.</p></td></tr><tr><td align="left"><p>Evidence</p></td><td align="left"><p>Findings from systematic reviews and meta-analyses have shown that CoCM is more effective than usual care in achieving positive mental health treatment outcomes (e.g., Archer et al., <xref ref-type="bibr" rid="bibr9">2012</xref>; Asarnow et al., <xref ref-type="bibr" rid="bibr8">2015</xref>; Richardson et al., <xref ref-type="bibr" rid="bibr97">2017</xref>; Thota et al., <xref ref-type="bibr" rid="bibr123">2012</xref>; Woltman et al., <xref ref-type="bibr" rid="bibr133">2012</xref>). Research supports CoCM's cost-effectiveness (Wright et al., <xref ref-type="bibr" rid="bibr131">2016</xref>) and care cost savings potential (Yu et al., <xref ref-type="bibr" rid="bibr133">2017</xref>).</p></td><td align="left"><p>Research has found that PBIS results in reductions in student problem behavior (Bradshaw et al., <xref ref-type="bibr" rid="bibr16">2010</xref>; Flannery et al., <xref ref-type="bibr" rid="bibr41">2014</xref>) and improved prosocial behavior (McIntosh et al., <xref ref-type="bibr" rid="bibr73">2011</xref>), academic achievement (Horner et al., <xref ref-type="bibr" rid="bibr54">2009</xref>; Kelm et al., <xref ref-type="bibr" rid="bibr59">2014</xref>), perceptions of school safety (Horner et al., <xref ref-type="bibr" rid="bibr54">2009</xref>) and school organization (Bradshaw et al., <xref ref-type="bibr" rid="bibr15">2009</xref>; Ross et al., <xref ref-type="bibr" rid="bibr98">2011</xref>).</p></td><td align="left"><p>SoC is effective in improving youth emotional and behavioral health, community and school functioning, and effective use of services, as well as in reducing caregiver strain (Stroul et al., <xref ref-type="bibr" rid="bibr114">2021</xref>); Suter & Bruns, <xref ref-type="bibr" rid="bibr121">2009</xref>). At the service systems and policy levels, SoC improves interagency collaboration, increases utilization of community supports, decreases use of institutional level supports, and yields cost savings (Stroul et al., <xref ref-type="bibr" rid="bibr117">2015</xref>)</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0175304773-6">Core Components of Exemplar Models</hd> <p>In synthesizing the literature describing the evidence and implementation of each exemplar model, we identified core components of multilevel service delivery frameworks via a multi-stage review and analysis process. The research team that undertook this work was composed of individuals with expertise in youth mental health services in one or more service delivery settings and implementation science. During the first stage, the team completed a review of representative literature for each framework, specifically relating to its evidence base, key features, and implementation considerations. At least two members of the team completed the review for each framework and developed syntheses regarding the given framework's guiding principles, key features, and discrete practices. Written syntheses were refined through discussion within pairs, as applicable. Then, the larger team engaged in group discussion to identify commonalities across frameworks and to reach consensus on core components that spanned each exemplar framework. This discussion centered the distillation of each framework's guiding principles, key features, and discrete practices, which were compared, contrasted, and labeled. Our goal was to identify frameworks' shared core components, recognizing that each framework may have additional unique features that may not be representative of the essential elements of another framework. The identified core components were revised and refined through additional discussion, which centered the labels applied to core components, potential areas of overlap, and areas for consolidation. We revisited the reviewed literature or technical assistance documentation during this process. Finally, we identified illustrative examples of each component as applied to each framework, and made additional revisions to component labels or definitions, as indicated.</p> <p>The resulting core components included the following: a) a systems-level approach, b) data-driven problem solving and decision-making, c) multiple levels of service intensity using evidence-based practices (EBPs), d) cross-linking service sectors, e) multiple providers[<reflink idref="bib1" id="ref73">1</reflink>] working together, including in teams, and f) built-in implementation strategies that facilitate delivery of the overall model. Consistent with the multilevel nature of these service delivery frameworks, the identified core components also include multilevel features (e.g., interventions provided at varied service levels/intensity, data collected at multiple levels [client, system], teams comprised of individuals who work at varied levels within their service setting/organization, etc.). Each core component is summarized in Table 2 and described below.</p> <p>Table 2 Core components of multilevel service delivery frameworks and examples illustrated for exemplar frameworks</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p><italic>Core component</italic> and definition</p></th><th align="left" colspan="3"><p>Exemplar framework</p></th></tr><tr><th align="left"><p>Collaborative care (CoCM)</p></th><th align="left"><p>Positive behavior interventions and supports (PBIS)</p></th><th align="left"><p>System of care (SoC)</p></th></tr></thead><tbody><tr><td align="left"><p><italic>Systems-level approach to organization and service delivery</italic></p><p>Structures and/or processes that enable service coordination within broader sociopolitical systems and engagement of actors across local, regional, and state policy, administrative, and provider levels of a system in order for the framework to be adopted/ implemented with high quality</p></td><td align="left"><p>Strategic redesign of service delivery is often needed to successfully integrate and deliver CoCM</p><p>Emphasizes population-based care for a defined group of patients</p><p>CoCM structures supported under new Medicaid reimbursement rules</p></td><td align="left"><p>Alignment of policies and procedures across classroom, grade, building, district, and state levels</p><p>Emphasizes a whole school/district approach, with involvement from all individuals in both classroom and non-classroom settings in a school building</p><p>Implementation of EBPs, systems, and associated data-based decision-making are adapted to the context of the local culture such that characteristics and cultural learning histories of contributors/collaborators, implementers, and consumers are embedded in a comprehensive and authentic manner</p></td><td align="left"><p>Requires action at the local, regional, and state levels, including to support cross-sector care coordination and service provision at the youth- and family-level</p><p>Practices for systems-level management and coordination</p><p>Incorporates elements of a population-based public health framework (promotion, prevention, screening, early identification and intervention, in addition to treatment)</p><p>Linkages between child-serving agencies and programs across administrative and funding boundaries</p></td></tr><tr><td align="left"><p><italic>Data-driven problem solving and decision-making</italic></p><p>Data, about clients or service delivery, are used to guide delivery of component Interventions and implementation and evaluation of frameworks</p></td><td align="left"><p>Measurement based care (i.e., treatment goals set to measurable targets, data used to monitor patient progress)</p><p>Relies upon validated instruments</p></td><td align="left"><p>Ongoing facilitation and use of a problem-solving process to support planning, implementing, and evaluating effectiveness of services</p><p>Comprehensive, efficient, and user-friendly data systems for supporting decision-making at all levels from the individual student level up to the aggregate district level</p><p>Continuous progress monitoring</p><p>Universal & comprehensive screening to inform service delivery</p><p>Fidelity of implementation measures</p></td><td align="left"><p>Use of a problem-solving process to support planning, implementing, and evaluating effectiveness of services</p><p>Outcome-based service provision</p><p>Family and youth benchmarks of functioning</p><p>Fidelity of implementation measures</p></td></tr><tr><td align="left"><p><italic>Multiple levels of service intensity using evidence-based practices (EBP)</italic></p><p>A public health approach to service delivery in which interventions are provided to youth/families at increasing levels of intensity (i.e., promotion, prevention, early intervention, intensive treatment) and using research-based practices</p></td><td align="left"><p>Screening, evaluation, and progress-monitoring data are used to guide decisions about level of stepped care; referrals provided to specialty care, social services, and community-based resources as needed</p><p>Focus from prevention to referral for intensive treatment</p><p>Intensity of treatment is augmented based on client response and in accordance with progress-monitoring data</p></td><td align="left"><p>Continuum of EBP</p><p>Includes supports at Tier 1 (universal), Tier 2, and Tier 3</p><p>An integrated and sequenced organization of practices is developed such that (a) a core curriculum is provided for all students, (b) modification of this core is arranged for students whose performance identified as nonresponsive, and (c) specialized and intensive curriculum is developed for students whose performance is deemed nonresponsive to the modified core</p><p>Elements of this continuum have empirical evidence to support efficacy, effectiveness, relevance and social validity, and durability</p></td><td align="left"><p>Cross-sector systems integration and care coordination at the community level</p><p>3 levels of service delivery (primary and secondary prevention, tertiary intervention)</p><p>Intensive family and youth-driven wraparound teams</p><p>Customized, individualized service delivery</p><p>All relying upon evidence-informed and promising practices, as well as interventions supported by practice-based evidence</p></td></tr><tr><td align="left"><p><italic>Cross-linking service sectors</italic></p><p>Professionals and resources from at least two service sectors (e.g., physical health and mental health) are coordinated for service provision</p></td><td align="left"><p>Physical health & mental health care</p><p>Referrals made to other sectors (specialty care, behavioral health, social services), as indicated by client need</p></td><td align="left"><p>Education and mental health (e.g., wraparound supports and culturally responsive person-centered planning that actively involves family and community supports and resources)</p></td><td align="left"><p>Key feature of SoC</p><p>Single plan of care per youth/family that is linked across children's mental health, child protection, juvenile justice, education, and community support service sectors</p><p>Linkages across administrative and funding boundaries</p></td></tr><tr><td align="left"><p><italic>Teaming/multiple providers working together</italic></p><p>Multiple providers with varied expertise work together to coordinate services (e.g., collaboratively developed treatment plans) in support of youth/family outcomes</p></td><td align="left"><p>Physical and mental health providers co-located</p><p>Clearly defined roles for each care team member</p><p>Collaboratively develop and share treatment plans that incorporate patient goals</p><p>Regular team meetings</p></td><td align="left"><p>Implementation of EBPs and systems are guided, coordinated, and administered by a local team comprised of representation from leadership, implementers, consumers (i.e., youth, families, peers), and content experts</p><p>Team ensures high intervention fidelity, management of resources, and data-based decision-making</p><p>Local personnel have high levels of content knowledge, fluency, and experience to support the culturally relevant and high-intervention-fidelity of EBPs and systems</p></td><td align="left"><p>Providers brought together at the youth- and family-level through wraparound; create an individualized service plan in partnership with child and family</p><p>Care management at the practice level to ensure coordination of services, and modifications based on changing needs</p><p>Community and state-level cross-sector collaboratives develop cross-system agreements and financing</p><p>Family and Youth Peer Support components that require workforce development unique to SoC</p></td></tr><tr><td align="left" colspan="4"><p><italic>Built-in Implementation Strategies</italic></p></td></tr><tr><td align="left"><p>Examples of implementation strategies <sup>a</sup> included as core components of the service delivery frameworks</p></td><td align="left" colspan="3" /></tr><tr><td align="left"><p> Train and educate stakeholders</p></td><td align="left"><p>✓ Conduct ongoing training, provide ongoing consultation (administrative, clinical)</p></td><td align="left"><p>✓Conduct ongoing training, provide ongoing consultation (coaching)</p></td><td align="left"><p>✓ Conduct ongoing training (workforce training), provide ongoing consultation (coaching)</p></td></tr><tr><td align="left"><p> Provide interactive assistance</p></td><td align="left"><p>✓ Centralize technical assistance (e.g., AIMS Center), Provide clinical supervision</p></td><td align="left"><p>✓ Facilitation/problem-solving, Provide local technical assistance, Centralize technical assistance (e.g., Center on PBIS)</p></td><td align="left"><p>✓ Provide local technical assistance, Centralize technical assistance (e.g., state-level)</p></td></tr><tr><td align="left"><p> Support clinicians</p></td><td align="left"><p>✓ Revise professional roles, Create new clinical teams</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p> Engage consumers</p></td><td align="left"><p>✓ Prepare patients/consumers to be active participants (client engagement via education, brief interventions, motivational support, linkages with community resources)</p></td><td align="left" /><td align="left"><p>✓ Increase demand (e.g., via strategic communication, outreach, advocacy)</p></td></tr><tr><td align="left"><p> Change infrastructure</p></td><td align="left"><p>✓ Change record systems (clinical information/data systems)</p></td><td align="left"><p>✓ Change record systems (clinical information/data systems), Develop local policy that supports implementation (alignment)</p></td><td align="left" /></tr><tr><td align="left"><p> Use financial strategies</p></td><td align="left"><p>✓ Place innovation on fee for service lists/formularies (Medicaid)</p></td><td align="left" /><td align="left"><p>✓ Access new funding (cross-sector funding linkages), Place innovation on fee for service lists/formularies (Medicaid)</p></td></tr><tr><td align="left"><p> Adapt and tailor to context</p></td><td align="left"><p>✓ Promote adaptability (adapt treatment if targets are not reached)</p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p> Use evaluative and iterative strategies</p></td><td align="left"><p>✓ Purposely reexamine the implementation (review provider and program outcomes to inform quality improvement actions)</p></td><td align="left"><p>✓ Assess for readiness and identify barriers and facilitators (emphasizes team-based readiness for implementation), Develop a formal implementation blueprint, Develop and organize quality monitoring system (fidelity monitoring)</p></td><td align="left"><p>✓ Develop and organize quality monitoring system (fidelity, quality, effectiveness, outcomes)</p></td></tr><tr><td align="left"><p> Selected references for further reading</p></td><td align="left"><p>Campo et al. (<xref ref-type="bibr" rid="bibr21">2005</xref>), Goodrich et al. (<xref ref-type="bibr" rid="bibr47">2013</xref>), Kolko and Perrin (<xref ref-type="bibr" rid="bibr64">2014</xref>)</p></td><td align="left"><p>McIntosh and Goodman (<xref ref-type="bibr" rid="bibr74">2016</xref>), Center on PBIS (<xref ref-type="bibr" rid="bibr24">2015</xref>)</p></td><td align="left"><p>Pires (<xref ref-type="bibr" rid="bibr91">2010</xref>), Stroul et al. (<xref ref-type="bibr" rid="bibr115">2010</xref>), Stroul et al. (<xref ref-type="bibr" rid="bibr114">2021</xref>)</p></td></tr></tbody></table> </ephtml> </p> <p> <emph>EBP</emph> evidence-based practice <sups>a</sups>To facilitate consistency in language (Proctor et al., [<reflink idref="bib95" id="ref74">95</reflink>]), implementation strategy examples are organized by categories (Waltz et al., [<reflink idref="bib128" id="ref75">128</reflink>]) and names articulated in the Expert Recommendations for Implementing Change (ERIC) study (Powell et al., [<reflink idref="bib94" id="ref76">94</reflink>]) and School Implementation Strategies, Translating ERIC Resources (SISTER) project (Cook et al., [<reflink idref="bib30" id="ref77">30</reflink>]; i.e., for PBIS, when applicable)</p> <hd id="AN0175304773-7">Systems-Level Approach to Organization and Service Delivery</hd> <p>Integral to each framework is a systems-level approach to the organization and delivery of mental health care to youth clients and their families. The systems-level approach describes structures and/or processes that enable service coordination within larger sociopolitical systems, enacting or clarifying policy at varied levels, and involvement and coordination across invested actors within a given service setting. As such, multilevel frameworks require engagement of individuals across levels within a system, including state and local policy makers, administrative decision makers, direct service providers (e.g., school, practice, behavioral health agency), community members, clients, and families, and consider the influence of social environments on human behavior. Drawing from bioecological systems theory (Bronfenbrenner & Ceci, [<reflink idref="bib18" id="ref78">18</reflink>]), a systems-level approach emphasizes that children are shaped by their interaction with others and their contexts (e.g., family, peers, connections between providers and family, state/federal policies related to mental health). Therefore, effectively delivering services within each framework requires augmenting organization and/or service-setting-level priorities, administrative workflow, and decision-making, as well as shaping provider and client interactions. For example, CoCM necessitates the strategic redesign of service delivery such that mental health care is integrated into primary care practice teams, data-systems, treatment planning and delivery, and provider accountability metrics (Goodrich et al., [<reflink idref="bib47" id="ref79">47</reflink>]). SoC requires integration of services within a behavioral health system and specifies practices for system-level management, coordination, and integrated care management (Stroul et al., [<reflink idref="bib115" id="ref80">115</reflink>]). Supportive infrastructure at the local, regional, and state levels enables cross-sector care coordination and service provision (Stroul et al., [<reflink idref="bib114" id="ref81">114</reflink>]). Each framework further prioritizes a population-based or public-health approach to service delivery, whether for a defined group of patients (CoCM), a whole school or district (PBIS), or children and families in the local community (SoC).</p> <hd id="AN0175304773-8">Data-Driven Problem-Solving and Decision-Making</hd> <p>As approaches to organizing and structuring a spectrum of circumscribed practices, data-driven problem solving and decision-making processes within multilevel frameworks have two primary purposes. First, client data are collected to guide prevention and treatment decisions. Second, implementation and service process and/or outcome data are collected to inform ongoing implementation and evaluation of the component interventions and the frameworks themselves. Client data collection to inform service delivery may include population-level mental health screening, diagnostic assessment (e.g., for those referred via screening), and intervention progress monitoring (Campo et al., [<reflink idref="bib21" id="ref82">21</reflink>]; Molloy et al., [<reflink idref="bib82" id="ref83">82</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref84">114</reflink>]). Implementation of each framework may be guided by fidelity data (i.e., intervention and strategy) and/or service delivery metrics (e.g., number of youths screened). In CoCM, for example, physicians may administer mental health screenings and refer youth to mental health specialists for further evaluation and triage. Clinical data then inform development of treatment protocols for each client. Treatment goals are aligned toward measurable targets and treatment is augmented based on progress-monitoring data (e.g., Goodrich et al., [<reflink idref="bib47" id="ref85">47</reflink>]; Lyon et al., [<reflink idref="bib72" id="ref86">72</reflink>]). Clinical information systems (e.g., electronic health records) and review of population-level data facilitate monitoring of care provision at the practice-level and evaluation of provider- and program-level outcomes to inform quality improvement (AIMS Center, [<reflink idref="bib2" id="ref87">2</reflink>]). Similarly, intervention fidelity data collected using instruments designed for PBIS (e.g., Tiered Fidelity Inventory) facilitates school teams' problem-solving processes that support planning, implementation, and evaluation of tiered services (Algozzine et al., [<reflink idref="bib6" id="ref88">6</reflink>]). Readers are referred to Bickman et al. ([<reflink idref="bib13" id="ref89">13</reflink>]) for a broader discussion of how data may be used to facilitate precision in mental health treatment.</p> <hd id="AN0175304773-9">Multiple Levels of Service Intensity Using EBPs</hd> <p>Within each exemplar multilevel service delivery framework, services are provided to youth and/or families at increasing levels of intensity. Each framework focuses on prevention, early identification and intervention, and intensive treatment, with providers using EBPs at each level to prevent or address known problems. For example, within PBIS, a continuum of EBPs—from universal (all students and staff in all settings) to targeted (some students) to intensive (few students with greatest need)—are layered across the school and grade/classroom levels (Center on PBIS, [<reflink idref="bib24" id="ref90">24</reflink>]). When implemented well, PBIS prioritizes prevention via high quality learning environments, including universal behavioral expectations and social-emotional learning and supports, for all students and staff and across all settings (i.e., school-wide, classroom, and non-classroom settings; Center on PBIS, [<reflink idref="bib24" id="ref91">24</reflink>]). Targeted and intensive interventions are focused, frequent, and small group or individually oriented, respectively. These interventions are provided in addition to universal interventions, with a goal of reducing the number and/or intensity of students' behavioral or social-emotional challenges (Center on PBIS, [<reflink idref="bib24" id="ref92">24</reflink>]).</p> <hd id="AN0175304773-10">Cross-Linking Service Sectors</hd> <p>Effectively changing systems to support data-based decision-making and EBP across multiple service tiers is dependent upon the cross-linking of service sectors and multiple providers with varied expertise working together within the local context. These two core components may be clearly differentiated for some exemplar multilevel service delivery frameworks (i.e., CoCM and SoC) which involve co-location and care coordination across service systems, but are less differentiated within other frameworks (e.g., PBIS). While we discuss each component separately, their interdependent nature is essential to keep in mind. Cross-linking of service sectors implies that professionals and resources from at least two service sectors are coordinated for service delivery. For example, within SoC, a single plan of care per family is developed, with care coordinated across sectors, including children's mental health services, child protection, juvenile justice, education, and community support sectors. This systems-level integration includes the creation of linkages between child-serving agencies and programs across administrative and funding boundaries (Stroul et al., [<reflink idref="bib115" id="ref93">115</reflink>]). This might be achieved by creating a "bundled" payment system that allows for enhanced care coordination, contracted services, and flexible spending. As an example, New Hampshire established a comprehensive Medicaid benefit (via 1951[i] mechanism; 2018), that supports the intensive wraparound model. Cross-sector linkages in SoC are further supported through SoC teams that discuss policy, systems, and practice-level changes, and that prioritize collaboration.</p> <hd id="AN0175304773-11">Multiple Providers Work Together</hd> <p>Within each framework, multiple providers with varied expertise work together intentionally to support youth and family outcomes, for example, via practice, care, or problem-solving teams. Common across each model are team-developed prevention, treatment or outcome plans with goals that are collaboratively generated. Multidisciplinary CoCM teams (e.g., primary care practitioner, mental health specialist, and care manager) include each team member operating within a clearly defined role (Campo et al., [<reflink idref="bib21" id="ref94">21</reflink>]; Lyon et al., [<reflink idref="bib72" id="ref95">72</reflink>]). For example, physicians may screen for mental health symptoms and make referrals to specialists who provide further evaluation whereas care managers may act as liaisons, provide brief intervention, and monitor clients' progress (e.g., Campo et al., [<reflink idref="bib21" id="ref96">21</reflink>]; Katon et al., [<reflink idref="bib58" id="ref97">58</reflink>]). PBIS teams include general educators, special educators, school mental health professionals and administrators who leverage collected data and formal problem-solving procedures to monitor and enhance implementation (McIntosh & Goodman, [<reflink idref="bib74" id="ref98">74</reflink>]). In SoC, effective wraparound practice occurs within a coordinated network led by the expressed needs and goals of the youth and family, including the care coordinator, supervisor/coach, administrators, peer supporters, community-based providers, and systems level partners (e.g., regional/state-level child protection authority), who each have specialized knowledge/skills (Miles et al., [<reflink idref="bib80" id="ref99">80</reflink>]; Schurer et al., [<reflink idref="bib108" id="ref100">108</reflink>]; Walker et al., [<reflink idref="bib127" id="ref101">127</reflink>]).</p> <hd id="AN0175304773-12">Built-In Implementation Strategies</hd> <p>Each multilevel framework also includes several practices, tools, or techniques—implementation strategies—that facilitate agencies' and/or contributors' implementation of the overall framework. Unlike stand-alone EBP, many implementation strategies tend to be "baked into" multilevel service delivery frameworks. Prominent examples of implementation strategies that are core within each framework are included in Table 2, with strategies organized by category and labeled with names provided in the Expert Recommendations for Implementing Change (ERIC) study (Powell et al., [<reflink idref="bib94" id="ref102">94</reflink>]; Waltz et al., [<reflink idref="bib128" id="ref103">128</reflink>]) or School Implementation Strategies, Translating ERIC Resources (SISTER) project (Cook et al., [<reflink idref="bib30" id="ref104">30</reflink>]).</p> <p>A variety of discrete strategies, applied with specific service settings in mind, are represented across categories. For example, <emph>training and education</emph> of respective contributors through initial and ongoing training/professional development and/or consultation/coaching in the target framework is prioritized (George et al., [<reflink idref="bib45" id="ref105">45</reflink>]; Kolko et al., [<reflink idref="bib63" id="ref106">63</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref107">114</reflink>]). Strategies broadly characterized by <emph>providing interactive assistance</emph> are also prominent and include clinical supervision in CoCM (e.g., case managers with psychiatrists), as well as regional, state, or national-level technical assistance (TA) centers for each framework (e.g., Advancing Integrated Mental Health Solutions [AIMS] Center for CoCM, National Center on PBIS, state/regional TA in SoC; Center on PBIS, [<reflink idref="bib24" id="ref108">24</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref109">114</reflink>]). Congruent with an emphasis on using data to guide decision-making, CoCM and PBIS emphasize changing record systems (<emph>change infrastructure</emph>), for example by modifying or implementing clinical/data information systems (e.g., Schoolwide Information System [SWIS]; Campo et al., [<reflink idref="bib21" id="ref110">21</reflink>]; PBISApps, [<reflink idref="bib90" id="ref111">90</reflink>]). <emph>Financial strategies</emph>, such as pursuing Medicaid reimbursement for services are often leveraged for CoCM (Goodrich et al., [<reflink idref="bib47" id="ref112">47</reflink>]; Wolk et al., [<reflink idref="bib132" id="ref113">132</reflink>]) and SoC (Stroul et al., [<reflink idref="bib114" id="ref114">114</reflink>]); however, we did not observe a similar focus within PBIS literature and documentation. This discrepancy may reflect differences in the financial structures of primary care and outpatient/specialty mental health settings in comparison to educational settings, such that some reimbursement models may not recognize schools as mental health services sites (Hoover & Bostic, [<reflink idref="bib52" id="ref115">52</reflink>]; see Schultz et al., [<reflink idref="bib107" id="ref116">107</reflink>] for policy recommendations). Finally, whereas <emph>evaluative and iterative strategies</emph> appear to be integrated within each framework, the specific strategies prioritized may vary depending on the framework and service setting. Systems for implementation quality monitoring are common across frameworks (Goodrich et al., [<reflink idref="bib47" id="ref117">47</reflink>]; Stroul et al., [<reflink idref="bib114" id="ref118">114</reflink>]), however, PBIS implementation guidance (Center on PBIS, [<reflink idref="bib24" id="ref119">24</reflink>]) emphasizes conducting needs assessments and developing a formal implementation blueprint prior to implementation. As described in the discussion, much additional work is indicated surrounding the identification, specification, and tailoring of the implementation strategies that tend to be most commonly integrated into multilevel frameworks in mental health.</p> <hd id="AN0175304773-13">Implementation Determinants for Core Components of Multilevel Service Delivery Frameworks</hd> <p>Although identifying core components of multilevel service delivery frameworks is important in assessing both intervention and implementation strategy fidelity (e.g., Lawson et al., [<reflink idref="bib68" id="ref120">68</reflink>]), research also indicates that the determinants that affect implementation of each component may vary (e.g., Wood et al., [<reflink idref="bib132" id="ref121">132</reflink>]). Identifying how implementation determinants interface with frameworks' core components and similarities in the determinants that are relevant across frameworks can facilitate future research that seeks to evaluate the impact of determinants on implementation outcomes as well as the identification, specification/tailoring, and testing of implementation strategies tailored to these determinants.</p> <p>Therefore, after identifying the core components of each exemplar model, we sought to map relevant implementation determinants, delineated in the implementation literature, to each multilevel service delivery framework component. We repeated the multistage review and analysis process described above with emphasis on implementation of each exemplar framework. Reviewed literature included empirical studies evaluating a given framework, so long as implementation was also addressed (e.g., evaluation or reporting of implementation outcomes, process evaluation describing implementation procedures and/or challenges and successes); implementation studies (e.g., that identified or investigated implementation determinants or strategies; implementation evaluations); and systematic review articles. Across reviewed literature, the identified determinants originated from study data or were extracted from the manuscript text (e.g., results section, discussion section).</p> <p>Identified determinants were subsequently coded using a determinant framework (Nilsen, [<reflink idref="bib86" id="ref122">86</reflink>])—the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., [<reflink idref="bib33" id="ref123">33</reflink>]). The CFIR is a widely-used, meta-theoretical framework that delineates a set of constructs, with definitions, that may influence outcomes of implementation efforts across settings (Damschroder et al., [<reflink idref="bib33" id="ref124">33</reflink>], [<reflink idref="bib34" id="ref125">34</reflink>]). Thus, the CFIR provides a structure for synthesizing determinants across studies and settings using a common terminology (Damschroder et al., [<reflink idref="bib33" id="ref126">33</reflink>]; Kirk et al., [<reflink idref="bib60" id="ref127">60</reflink>]). Four CFIR domains were of focus, including: the outer setting (e.g., policies and incentives, inter-organizational networks), inner setting (e.g., intra-organization networks and communication, leadership, access to information), individual (e.g., knowledge and beliefs, self-efficacy), and intervention (e.g., complexity, cost) levels. For the purposes of this study, we defined the outer setting as the macro-economic, sociopolitical, and service-system context (e.g., state or local education agency, health care system); the inner setting as the more proximal features of a given service setting (e.g., school, practice); and the intervention level as encompassing framework-specific features.</p> <p>Coding was led by the first author, who used the published definitions of CFIR constructs, supplementary material, and inclusionary and exclusionary criteria described in the CFIR codebook (CFIR, [<reflink idref="bib29" id="ref128">29</reflink>]; Damschroder et al., [<reflink idref="bib33" id="ref129">33</reflink>]) to determine code labels. The preliminary codes were reviewed by the remaining authors, who each hold expertise in exemplar service delivery framework(s) and/or implementation science. Determinants were recoded as indicated by feedback throughout the coding process. Aligned with our stated goals for this study, we aimed to highlight implementation determinants that were evident across multiple sources for each exemplar framework so as to be a catalyst for future work that empirically examines how relevant determinants impact implementation outcomes as well as to inform selection of context-specific implementation strategies (beyond those that are baked-in) to improve these frameworks' outcomes (Powell et al., [<reflink idref="bib93" id="ref130">93</reflink>]). Thus, this process was not intended to be systematic nor exhaustive; a comprehensive review was beyond the scope of this paper but could be an important subsequent step (see Limitations and Future Directions). In the sections below and in Table 3, we present select examples of implementation determinants, organized by their relation to each core component presented previously. Determinants were identified at each contextual level.</p> <p>Table 3 Selected implementation determinants for core components of exemplar multilevel service delivery frameworks and relation to CFIR domains and constructs</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>Core component</p></th><th align="left" colspan="3"><p>Exemplar framework</p></th></tr><tr><th align="left"><p>Collaborative care (CoCM)</p></th><th align="left"><p>Positive behavioral interventions and supports (PBIS)</p></th><th align="left"><p>System of care (SoC)</p></th></tr></thead><tbody><tr><td align="left"><p>Systems level approach</p></td><td align="left"><p>Financial barriers (e.g., reimbursement challenges; Solberg et al., <xref ref-type="bibr" rid="bibr111">2013</xref>; Sanchez et al., <xref ref-type="bibr" rid="bibr103">2010</xref>)—<bold>Intervention</bold><sup><bold>a</bold></sup><bold> (Cost), Outer Setting (External Policies & Incentives)</bold></p></td><td align="left"><p>Funding (Kincaid et al., <xref ref-type="bibr" rid="bibr61">2007</xref>) and financial resources (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Intervention (Cost), Outer Setting (External Policies & Incentives)</bold></p></td><td align="left"><p>Financial support following grant periods (Vinson et al., <xref ref-type="bibr" rid="bibr125">2001</xref>)—<bold>Intervention (Cost), Outer Setting (External Policies & Incentives)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Leadership support (i.e., strong organizational leadership; Whitebird et al., <xref ref-type="bibr" rid="bibr130">2014</xref>)—<bold>Inner Setting (Leadership Engagement)</bold></p></td><td align="left"><p>School administrator (e.g., principal) support (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>; Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>); effective and supportive leadership (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Inner Setting (Leadership Engagement)</bold></p></td><td align="left"><p>Strong/committed and consistent leadership support (Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>; Stroul & Manteuffel, <xref ref-type="bibr" rid="bibr119">2007</xref>)—<bold>Inner Setting (Leadership Engagement)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Full-time staff dedicated to implementation (Wolk et al., <xref ref-type="bibr" rid="bibr132">2021</xref>)—<bold>Inner Setting (Available Resources, Access to Knowledge and Information)</bold></p></td><td align="left"><p>Receiving and maintaining support from school faculty and staff (Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>); Staff beliefs that conflict with PBIS philosophy or lack of buy in, limited to commitment to framework (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Characteristics of Individuals (Knowledge and Beliefs about the Intervention)</bold></p></td><td align="left"><p>Shared vision among contributors /leadership team (e.g., about systems change goals; Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>; Heflinger, <xref ref-type="bibr" rid="bibr50">1996</xref>; Mendenhall & Frauenholtz, <xref ref-type="bibr" rid="bibr78">2014</xref>)—<bold>Inner Setting (Relative Priority)</bold></p></td></tr><tr><td align="left" /><td align="left" /><td align="left"><p>District support (Kincaid et al., <xref ref-type="bibr" rid="bibr61">2007</xref>)—<bold>Outer Setting (External Policies & Incentives)</bold></p></td><td align="left"><p>Central entity/organization that can connect service providers/agencies (Heflinger, <xref ref-type="bibr" rid="bibr50">1996</xref>; Vinson et al., <xref ref-type="bibr" rid="bibr125">2001</xref>)—<bold>Inner Setting (Structural Characteristics)</bold></p></td></tr><tr><td align="left" /><td align="left" /><td align="left"><p>Opportunities for quality team and staff trainings (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>; Kincaid et al., <xref ref-type="bibr" rid="bibr61">2007</xref>; Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>)—<bold>Inner Setting (Available Resources)</bold></p></td><td align="left"><p>Commitment, support, priority from macro/higher-level policy and decision makers at state and local levels (e.g., to provide funding, express priority, support compliance, provide funding; Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>; Lunn et al., <xref ref-type="bibr" rid="bibr71">2011</xref>; Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>; Stroul & Manteuffel, <xref ref-type="bibr" rid="bibr119">2007</xref>)—<bold>Outer Setting (External Policies & Incentives)</bold></p></td></tr><tr><td align="left" /><td align="left" /><td align="left"><p>Multiple schools within a district implementing PBIS with fidelity and number of schools beginning implementation at the same time (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Outer Setting (Cosmopolitanism)</bold></p></td><td align="left" /></tr><tr><td align="left" /><td align="left" /><td align="left"><p>Lack of access to needed resources (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Inner Setting (Available Resources)</bold></p></td><td align="left" /></tr><tr><td align="left"><p>Data-driven problem solving and decision-making</p></td><td align="left"><p>Limited familiarity/understanding of how to implement this component of CoCM (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Readiness for Implementation, Access to Information and Knowledge), Individual (Knowledge and Beliefs about the Intervention)</bold></p></td><td align="left"><p>Administrator support and input (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>; McIntosh et al., <xref ref-type="bibr" rid="bibr76">2013</xref>)—<bold>Inner Setting (Readiness for Implementation, Leadership Engagement)</bold></p></td><td align="left"><p>Limitations in data available that can support decision-making or implementation evaluations (Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>; Suter & Bruns, <xref ref-type="bibr" rid="bibr121">2009</xref>)—<bold>Inner Setting (Readiness for Implementation, Available Resources)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Challenges with data/information systems (e.g., building infrastructure, integrating into practice, duplication, access limitations, inefficiency; Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>; Wolk et al., <xref ref-type="bibr" rid="bibr132">2021</xref>)—<bold>Inner Setting (Readiness for Implementation,Available Resources)</bold></p></td><td align="left"><p>Effective collection and use of data (e.g., implementation/service or client outcome data), including sharing data within and outside the school (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>; Kincaid et al., <xref ref-type="bibr" rid="bibr61">2007</xref>; Mcintosh et al., <xref ref-type="bibr" rid="bibr76">2013</xref>) and frequency of data sharing with whole school (McIntosh et al., <xref ref-type="bibr" rid="bibr75">2015</xref>)—<bold>Inner Setting (Readiness for Implementation, Available Resources)</bold></p></td><td align="left"><p>Understanding importance of evaluation and data in driving services (Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>)—<bold>Inner Setting (Access to Knowledge and Information)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Absent progress monitoring procedures (Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>)—<bold>Intervention (Design Quality and Packaging), Inner Setting (Compatibility)</bold></p></td><td align="left"><p>Regularly sharing data regarding implementation and its effects (Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>)—<bold>Process (Reflecting & Evaluating)</bold></p></td><td align="left"><p>Methods for sharing data in real-time about system trends and to inform data-based decision-making (Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>)—<bold>Inner Setting (Readiness for Implementation, Available Resources)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Staff knowledge (understanding) and confidence regarding intervention expectations, data collection/screening procedures, etc. (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Intervention (Design quality and packaging, Complexity), Inner Setting (Readiness for implementation, Access to information and knowledge), Individual (Knowledge and beliefs about the intervention)</bold></p></td><td align="left"><p>Staff buy-in (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>; McIntosh et al., <xref ref-type="bibr" rid="bibr76">2013</xref>)—<bold>Characteristics of Individuals (Knowledge and Beliefs about the Intervention)</bold></p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>Screening connected to standardized care pathway, inclusive of high-quality materials and training (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Intervention (Design Quality and Packaging), Inner Setting (Access to Knowledge and Information)</bold></p></td><td align="left" /><td align="left" /></tr><tr><td align="left" /><td align="left"><p>Quality of data collection tools (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Intervention (Design Quality and Packaging, Evidence Strength and Quality)</bold></p></td><td align="left" /><td align="left" /></tr><tr><td align="left"><p>Multiple levels of service intensity using evidence-based practice</p></td><td align="left"><p>Systematic monitoring of patients and follow-up (i.e., access to/availability of information, or efforts to set up; Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>; Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Readiness for Implementation, Access to Knowledge and Information, Compatibility)</bold></p></td><td align="left"><p>Resources, including time and for training, to support interventions, particularly at Tier 3 (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Inner Setting (Available Resources)</bold></p></td><td align="left"><p>Difficulty establishing shared service plans (i.e., due to challenges with cross-sector teams/collaboration; Mendenhall & Frauenholtz, <xref ref-type="bibr" rid="bibr78">2014</xref>; Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>)—<bold>Inner Setting (Networks and Communications), Intervention (Complexity)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Limited referral sites for patient's requiring referral due to high level of need (Wolk et al., <xref ref-type="bibr" rid="bibr132">2021</xref>)—<bold>Inner Setting (Readiness for Implementation, Available Resources), Outer Setting (Cosmopolitanism)</bold></p></td><td align="left"><p>Individualizing interventions at each service level, for example, using more behavior management tools during intervention (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>)—<bold>Intervention (Adaptability)</bold></p></td><td align="left"><p>Lack of professionals with training to provide specific types of services within the full service array (Heflinger, <xref ref-type="bibr" rid="bibr50">1996</xref>)—<bold>Inner Setting (Readiness for Implementation; Available Resources)</bold></p></td></tr><tr><td align="left" /><td align="left" /><td align="left" /><td align="left"><p>Lack of understanding of SoC approach and how to put it into practice, why important to change current practices to provide full service array (Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>)—<bold>Intervention (Relative Advantage), Inner Setting (Access to Knowledge and Information)</bold></p></td></tr><tr><td align="left"><p>Cross-linking service sectors</p></td><td align="left"><p>Physician/physical health provider discomfort with diagnosing/treating mental health problems (psychiatric treatment; Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>; Wolk et al., <xref ref-type="bibr" rid="bibr132">2021</xref>)—<bold>Inner Setting (Readiness for implementation, Access to Information and Knowledge), Individuals (Self-Efficacy)</bold></p></td><td align="left"><p>Prioritization/alignment with current approaches (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Inner Setting (Implementation Climate—Compatibility)</bold></p></td><td align="left"><p>Differences in service philosophy across agencies/agencies' limited understanding of one another's role, mandates, scope of services, service limitations all of which can challenge communication and relationships, and create conflict (Heflinger , <xref ref-type="bibr" rid="bibr50">1996</xref>; Mendenhall & Frauenholtz, <xref ref-type="bibr" rid="bibr78">2014</xref>; Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>)—<bold>Outer Setting (Cosmopolitanism), Inner Setting (Networks and Communications)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Co-location of providers from different service sectors (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Structural Characteristics, Networks and Communications), Outer Setting (Cosmopolitanism)</bold></p></td><td align="left"><p>Use of experts from within and outside of the school to offer training and explain PBIS benefits (Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>)—<bold>Inner Setting (Available Resources), Outer Setting (Cosmopolitanism)</bold></p></td><td align="left"><p>Insufficient commitment across child-serving systems (i.e., some agencies being more difficult to engage or are less willing to share resources; Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>)—<bold>Inner Setting (Relative Priority)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Integrated information systems that facilitate sharing of notes & messages (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>); potential concerns about physician autonomy and data privacy (Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>)—<bold>Inner Setting (Networks and Communications, Compatibility)</bold></p></td><td align="left" /><td align="left"><p>Shared values among partners (Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>)—<bold>Inner Setting (Relative Priority)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Provider shortages (Sanchez et al., <xref ref-type="bibr" rid="bibr103">2010</xref>)—<bold>Inner Setting (Access to Knowledge and Information, Available Resources)</bold></p></td><td align="left" /><td align="left"><p>Entities work collaboratively (Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>)—<bold>Outer Setting (Cosmopolitanism)</bold></p></td></tr><tr><td align="left"><p>Teaming/multiple providers working together</p></td><td align="left"><p>Breakdowns in networks and communication pathways (e.g., avoiding communication, using jargon, limited technology to support timely communication; Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>); Enhanced inter-provider communication, integrated information systems multidisciplinary meetings (Wolk et al., <xref ref-type="bibr" rid="bibr132">2021</xref>; Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Networks and Communications)</bold></p></td><td align="left"><p>Effective team communication and use of team meetings; team composition is representative (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>); Issues related to team functioning and communication (Kincaid et al.; <xref ref-type="bibr" rid="bibr61">2007</xref>), including lack of consistency (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>)—<bold>Inner Setting (Networks and Communications)</bold></p></td><td align="left"><p>Formalizing communication (Vinson et al., <xref ref-type="bibr" rid="bibr125">2001</xref>) and communication challenges (e.g., decision-making conversations not inclusive of all contributors; Mendenhall & Frauenholtz, <xref ref-type="bibr" rid="bibr78">2014</xref>)—<bold>Inner Setting (Networks and Communication), Intervention (Complexity)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Staff turnover (Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>)—<bold>Inner Setting (Structural Characteristics, Available Resources)</bold></p></td><td align="left"><p>High staff turnover (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Inner Setting (Structural Characteristics)</bold></p></td><td align="left"><p>Differing priorities among contributor s and insufficient commitment across clinicians, care managers, and their organizations (Heflinger, <xref ref-type="bibr" rid="bibr50">1996</xref>; Stroul, <xref ref-type="bibr" rid="bibr113">2013</xref>)—<bold>Inner Setting (Relative Priority)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Organizational culture and readiness for change (e.g., impacting attitudes, changes in practice, integration of mental health care, sharing care of patients; buy-in; Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Implementation Climate, Relative Priority, Culture, Readiness for Implementation, Access to Knowledge and Information)</bold></p></td><td align="left"><p>Staff trainings (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>); Need for technical assistance in data collection and recording, soliciting administrative support, and monitoring intervention implementation (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>)—<bold>Inner Setting (Available Resources)</bold></p></td><td align="left"><p>Co-location, facilitating active collaboration (Vinson et al., <xref ref-type="bibr" rid="bibr125">2001</xref>)—<bold>Inner Setting (Structural Characteristics, Networks and Communications)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Clearly defined roles (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>); Lack of agreement about responsibility for care (Sanchez et al., <xref ref-type="bibr" rid="bibr103">2010</xref>)—<bold>Intervention (Complexity), Inner Setting (Culture)</bold></p></td><td align="left"><p>Inadequate parental support (Chitiyo & Wheeler, <xref ref-type="bibr" rid="bibr28">2009</xref>), difficulty generating student involvement in implementation (Flannery et al., <xref ref-type="bibr" rid="bibr40">2009</xref>)—<bold>Inner Setting (Relative Priority)</bold></p></td><td align="left"><p>Strong, trusting working relationship (Behar & Hydaker, <xref ref-type="bibr" rid="bibr12">2009</xref>)—<bold>Inner Setting (Networks and Communication)</bold></p></td></tr><tr><td align="left" /><td align="left"><p>Clear leadership structure led by providers across service sector (Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)—<bold>Inner Setting (Leadership Engagement)</bold></p></td><td align="left"><p>Staff knowledge and awareness of relevant theories/activities (Fox et al., <xref ref-type="bibr" rid="bibr43">2022</xref>)—<bold>Characteristics of Individuals (Knowledge and Beliefs about the Intervention), Inner Setting (Readiness for Implementation, Access to Knowledge and Information)</bold></p></td><td align="left" /></tr><tr><td align="left" /><td align="left"><p>Co-location (Overbeck et al., <xref ref-type="bibr" rid="bibr88">2016</xref>; Wood et al., <xref ref-type="bibr" rid="bibr132">2017</xref>)- <bold>Inner Setting (Structural Characteristics)</bold></p></td><td align="left" /><td align="left" /></tr></tbody></table> </ephtml> </p> <p>Bold text depicts determinant domains and constructs (the latter specified using parentheses) using the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., [<reflink idref="bib33" id="ref131">33</reflink>]) <sups>a</sups>Intervention factors indicate features related to the multilevel service delivery framework or its component parts</p> <hd id="AN0175304773-14">Systems Level Approach</hd> <p>Determinants of implementation of the systems level approach, unsurprisingly, are often situated within the outer and inner settings. For example, within the outer setting, <emph>external policies and incentives</emph> include explicit support from higher-level policy and decision makers at state or local levels who can place priority on implementation and support compliance with enacted policies (Stroul & Manteuffel, [<reflink idref="bib119" id="ref132">119</reflink>]). Supportive policies that provide funding or facilitate reimbursement for component interventions may counter financial barriers to implementation (e.g., insufficient funding following grant award periods; Stroul, [<reflink idref="bib113" id="ref133">113</reflink>]; Vinson et al., [<reflink idref="bib125" id="ref134">125</reflink>]). Within the inner setting, strong organizational leadership (i.e., <emph>leadership engagement</emph>) can guide implementation of the multilevel service delivery framework across the system—whether that is a primary care practice, local school or school district, or community behavioral health agency (Fox et al., [<reflink idref="bib43" id="ref135">43</reflink>]; Stroul, [<reflink idref="bib113" id="ref136">113</reflink>]; Whitebird et al., [<reflink idref="bib130" id="ref137">130</reflink>]). Developing a shared vision and goals for implementation across implementers within the organization is also a prominent determinant of this component (i.e., <emph>relative priority</emph>; Heflinger, [<reflink idref="bib50" id="ref138">50</reflink>]; Mendenhall & Frauenholtz, [<reflink idref="bib78" id="ref139">78</reflink>]). Access to adequate <emph>resources</emph>, such as full-time staff who support implementation (Wolk et al., [<reflink idref="bib132" id="ref140">132</reflink>]) and frequent opportunities for quality training and ongoing technical assistance, also appear important (Flannery et al., [<reflink idref="bib40" id="ref141">40</reflink>]; Fox et al., [<reflink idref="bib43" id="ref142">43</reflink>]).</p> <hd id="AN0175304773-15">Data-Driven Problem Solving and Decision-Making</hd> <p>Inner setting factors, including <emph>readiness for implementation</emph> and <emph>available resources</emph>, were similarly prominent for data-driven problem solving and decision-making. For example, providers' use of client data to triage clients to the appropriate level of care or guide treatment decisions, and implementation/service data to evaluate implementation progress may be influenced by the extent to which data/information systems have been established and by providers' understanding of how to use data for these purposes (e.g., Overbeck et al., [<reflink idref="bib88" id="ref143">88</reflink>]; Wood et al., [<reflink idref="bib132" id="ref144">132</reflink>]). For example, developing data systems to be able to manage and efficiently report data collected for multiple purposes (e.g., client, intervention and strategy fidelity, and service data, with multiple potential sources/measures for each type of data) is important for effectively using data to guide service delivery and implementation (e.g., Center on PBIS, [<reflink idref="bib26" id="ref145">26</reflink>]). When insufficient data are available—such as limited intervention fidelity (Suter & Bruns, [<reflink idref="bib121" id="ref146">121</reflink>]) or outcome data (Stroul, [<reflink idref="bib113" id="ref147">113</reflink>])—decision-making and evaluation of interventions may be challenging. Providers' knowledge and access to information about data systems, including how data align with frameworks' intervention expectations and procedures, are also salient (e.g., <emph>access to knowledge and information;</emph> Behar & Hydaker, [<reflink idref="bib12" id="ref148">12</reflink>]; Wood et al., [<reflink idref="bib132" id="ref149">132</reflink>]). Thus, attending to <emph>individuals' knowledge and beliefs</emph> about the service delivery framework, for example by building buy in and providing training on the importance of and effective use of data to guide decisions, can be helpful (Chitiyo & Wheeler, [<reflink idref="bib28" id="ref150">28</reflink>]; Flannery et al., [<reflink idref="bib40" id="ref151">40</reflink>]; McIntosh et al., [<reflink idref="bib76" id="ref152">76</reflink>]; Wolk et al., [<reflink idref="bib132" id="ref153">132</reflink>]). Several characteristics of the frameworks themselves may also impact data-use, including how data-systems and progress-monitoring procedures are connected to other components of the framework (i.e., <emph>design quality and packaging</emph>), availability of reliable/valid data collection tools (i.e., <emph>evidence strength and quality</emph>), and the <emph>complexity</emph> inherent in collecting and using multiple forms of data (Center on PBIS, [<reflink idref="bib26" id="ref154">26</reflink>]; Wood et al., [<reflink idref="bib132" id="ref155">132</reflink>]). For example, in PBIS, school teams are responsible for collecting and evaluating student outcome and fidelity (intervention and strategy) data to make decisions about school, classroom, and individual service delivery (e.g., which interventions are appropriate, whether to continue interventions, etc.). These decisions are further complicated by the need to simultaneously consider multiple sources of data to identify and act upon one of multiple plausible solutions that are appropriate for student(s) needs and a given school's context (Chaparro et al., [<reflink idref="bib27" id="ref156">27</reflink>]).</p> <hd id="AN0175304773-16">Multiple Levels of Service Intensity Using EBPs</hd> <p>Determinants for delivering EBPs with increasing levels of intensity included multiple dimensions of an organization's inner setting, such as <emph>available resources</emph>, <emph>readiness for implementation</emph>, and providers' <emph>access to knowledge and information</emph>. Select outer setting and intervention/framework components were also discussed, although less frequently. Seemingly beneficial to framework implementation is access to internal staff resources (i.e., inner setting; <emph>available resources</emph>, which contribute to <emph>readiness for implementation</emph>) as well as connections with external service providers (i.e., outer setting; <emph>cosmopolitanism</emph>—also see below) to facilitate referrals of clients with the highest levels of need (i.e., for CoCM and PBIS; Fox et al., [<reflink idref="bib43" id="ref157">43</reflink>]; Wolk et al., [<reflink idref="bib132" id="ref158">132</reflink>]) or to establish shared service plans (i.e., for SoC; Mendenhall & Frauenholtz, [<reflink idref="bib78" id="ref159">78</reflink>]; Stroul, [<reflink idref="bib113" id="ref160">113</reflink>]). However, provider shortages across care systems (Goforth et al., [<reflink idref="bib46" id="ref161">46</reflink>]; Health Resources and Services Administration, [<reflink idref="bib49" id="ref162">49</reflink>]) create barriers to offering the full service array (Heflinger, [<reflink idref="bib50" id="ref163">50</reflink>]). Importantly, the external (i.e., outer setting) factors contributing to the noted shortage of providers or referral sites (e.g., availability of graduate education programs/faculty to provide training, regional differences in provider availability and training opportunities, factors impacting retention, etc.; e.g., National Association of School Psychologists, [<reflink idref="bib84" id="ref164">84</reflink>]) were not a focus in the selected literature we reviewed. This seeming omission parallels the historically more limited focus on policy in implementation research, and thus is an important area for continued investigation (Crable et al., [<reflink idref="bib32" id="ref165">32</reflink>]). Finally, provider understanding of the service delivery framework and how to implement it (i.e., <emph>access to knowledge and information</emph>), including the degree to which the framework is perceived to be adaptable to needs of the local context and advantageous over current practice (i.e., <emph>adaptability</emph>), may impact whether current practices are augmented to support provision of the full service array (Chitiyo & Wheeler, [<reflink idref="bib28" id="ref166">28</reflink>]; Stroul, [<reflink idref="bib113" id="ref167">113</reflink>]).</p> <hd id="AN0175304773-17">Cross-Linking Service Sectors</hd> <p>Coordination of professionals and resources across service sectors may be influenced by factors that impact intra-organization <emph>networks and communication</emph>, the degree to which the organization is connected to other organizations within the local community (i.e., <emph>cosmopolitanism</emph>), and the extent to which this type of coordination is compatible with ongoing service provision and providers' perceptions of their role (i.e., <emph>compatibility)</emph>. For example, co-location of providers from different sectors (e.g., primary care and mental health in CoCM; <emph>structural characteristics</emph>) and integrated information systems (i.e., <emph>networks and communications</emph>) may facilitate collaboration across service sectors that enables service delivery (Wood et al., [<reflink idref="bib132" id="ref168">132</reflink>]). However, providers (e.g., primary care physicians) or service systems (e.g., schools) whose role/mission has traditionally not involved supporting child mental health may be hesitant to engage in related interventions and resistant to the collaboration required for implementation of multilevel service delivery frameworks to be successful (e.g., <emph>self-efficacy</emph>, <emph>compatibility</emph>; Fox et al., [<reflink idref="bib43" id="ref169">43</reflink>]; Overbeck et al., [<reflink idref="bib88" id="ref170">88</reflink>]; Wolk et al., [<reflink idref="bib132" id="ref171">132</reflink>]). Further, time spent in cross-system collaboration is typically not reimbursable. When care is coordinated across multiple agencies, such as in SoC, differences in service philosophy and gaps in understanding related to the roles and functions of partner agencies can challenge communication, strain relationships, and create conflict (e.g., <emph>networks and communications</emph>, <emph>cosmopolitanism</emph>; Heflinger, [<reflink idref="bib50" id="ref172">50</reflink>]; Mendenhall & Frauenholtz, [<reflink idref="bib78" id="ref173">78</reflink>]; Stroul, [<reflink idref="bib113" id="ref174">113</reflink>]). Thus, the degree to which the multilevel service delivery framework is aligned with the provider's or agency's larger mission and ongoing efforts is likely important for coordination (i.e., <emph>compatibility</emph>; Fox et al., [<reflink idref="bib43" id="ref175">43</reflink>]). Together, prominent determinants to effective cross-linking of service sectors suggest that co-location may be insufficient for effective care coordination, but rather that a shared vision (e.g., <emph>relative priority</emph>) and proximal (e.g., <emph>networks and communications</emph>) and distal systems (e.g., <emph>cosmopolitanism</emph>) to support effective communication and collaboration are also needed (Behar & Hydaker, [<reflink idref="bib12" id="ref176">12</reflink>]).</p> <hd id="AN0175304773-18">Multiple Providers Work Together</hd> <p>The degree to which multiple providers work together (e.g., via teaming) to support youth and family outcomes may also be impacted by provider <emph>networks and communication</emph> pathways. When these breakdown (e.g., jargon, ineffective communication) or formal communication systems (e.g., integrated information systems) are absent, provider collaboration can be impeded (Kincaid et al., [<reflink idref="bib61" id="ref177">61</reflink>]; Mendenhall & Frauenholtz, [<reflink idref="bib78" id="ref178">78</reflink>]; Wood et al., [<reflink idref="bib132" id="ref179">132</reflink>]). Communication across care team members may be enhanced through integrated information systems, multidisciplinary meetings, and fostering strong working relationships (Behar & Hydaker, [<reflink idref="bib12" id="ref180">12</reflink>]; Wolk et al., [<reflink idref="bib132" id="ref181">132</reflink>]). Similarly, provider collaboration may be supported when teams are representative, members feel included, and roles and responsibilities for care are clearly defined—however, these may also be challenged by the <emph>complexity</emph> inherent within multilevel service delivery frameworks (Fox et al., [<reflink idref="bib43" id="ref182">43</reflink>]; Sanchez et al., [<reflink idref="bib103" id="ref183">103</reflink>]). High rates of staff turnover (i.e., <emph>structural characteristics</emph>; Fox et al., [<reflink idref="bib43" id="ref184">43</reflink>]; Overbeck et al., [<reflink idref="bib88" id="ref185">88</reflink>]), providers' perceptions about the importance of multilevel service delivery framework implementation (i.e., <emph>knowledge and beliefs about the intervention</emph>), and shared commitment among providers, agencies, and youth/families (i.e., <emph>relative priority</emph>) may impact care coordination (Chitiyo & Wheeler, [<reflink idref="bib28" id="ref186">28</reflink>]; Flannery et al., [<reflink idref="bib40" id="ref187">40</reflink>]; Heflinger, [<reflink idref="bib50" id="ref188">50</reflink>]; Stroul, [<reflink idref="bib113" id="ref189">113</reflink>]). Organizational readiness for change (supported by engaged leaders; i.e., <emph>readiness for implementation</emph>, <emph>leadership engagement</emph>), provider's access to information/support for team-based care activities (i.e., <emph>access to knowledge and information</emph>), and provider attitudes about engaging in these practices (i.e., <emph>knowledge and beliefs about the intervention</emph>) also appear relevant (Chitiyo & Wheeler, [<reflink idref="bib28" id="ref190">28</reflink>]; Fox et al., [<reflink idref="bib43" id="ref191">43</reflink>]; Wood et al., [<reflink idref="bib132" id="ref192">132</reflink>]).</p> <hd id="AN0175304773-19">Discussion</hd> <p>Multilevel service delivery frameworks are championed as best-practice for meeting a variety of youth mental health needs and are increasingly adopted in primary youth mental health service settings (i.e., schools, primary care, outpatient/specialty mental health). Although exemplar frameworks are implemented within each service setting, their availability and quality are challenged by several factors including, but not limited to, a diffuse research base examining implementation (compared to discrete interventions). The siloed nature of the extant implementation research occurring within each setting has led to a limited understanding of the frameworks' common core components and their respective implementation determinants, knowledge of which could promote generalization across disciplines and inform future identification of implementation strategies.</p> <p>We described six core components shared by three exemplar multilevel service delivery frameworks—PBIS in schools, CoCM in primary care, and SoC in outpatient/specialty mental health settings. Multilevel service delivery frameworks require systems-level engagement that enables development of data and problem-solving systems to facilitate a graduated approach to service provision. Professionals and resources are coordinated across service sectors within an organization or community, supported by team-based care that guides service provision. Also built into these service delivery frameworks are implementation strategies that, for example, support changes to local infrastructure, emphasize training and education about these complex frameworks, and prioritize iterative evaluation and implementation.</p> <p>In articulating these common essential features of child mental health service delivery frameworks, this work begins to lay a conceptual foundation for multilevel service delivery that can inform research and practice across child and adult mental health service settings. This cross-sector foundation can enable information sharing and collaboration among implementation researchers, system leaders, and/or implementation intermediaries to advance the high-quality implementation of multilevel service delivery frameworks across youth mental health service settings, which is aligned with increasing calls for collaboration across youth mental health providers (The White House, [<reflink idref="bib122" id="ref193">122</reflink>]). For example, technical assistance providers (e.g., Center on PBIS) and district/school-based PBIS leadership teams may learn from the successes and challenges that primary care agencies encountered with data-driven problem-solving and decision-making systems and analyze whether the approach used in primary care could be adapted to fit their local context. Practical implementation reports and research that further articulates the ways in which data-informed delivery of multilevel frameworks in each setting can facilitate this type of interdisciplinary knowledge sharing. This foundation may further support the application and testing of multilevel frameworks in novel settings, such as CoCM within schools (Lyon et al., [<reflink idref="bib72" id="ref194">72</reflink>]) or PBIS in juvenile justice settings (Kumm et al., [<reflink idref="bib65" id="ref195">65</reflink>]), and may generalize to multilevel service delivery frameworks implemented within adult mental health care settings (e.g., CoCM, behavioral health crisis care systems; Goodrich et al., [<reflink idref="bib47" id="ref196">47</reflink>]; SAMHSA, [<reflink idref="bib120" id="ref197">120</reflink>]).</p> <p>Future implementation research can build upon this work. For example, future research can examine how intervention fidelity varies by core component, as well as how core components and their implementation relate to service outcomes (Blase & Fixsen, [<reflink idref="bib14" id="ref198">14</reflink>]). This body of work will complement research that examines the process for implementing multilevel service delivery frameworks' core features. For example, the Stages of Implementation Completion (SIC) is a measure of implementation processes that yields information about the time spent in varied implementation activities, the proportion of activities completed, and how much of the implementation process was achieved (Saldana et al., [<reflink idref="bib100" id="ref199">100</reflink>]). The SIC has been adapted and applied to CoCM in rural primary care clinics and was able to accurately assess implementation effectiveness and provide information on what challenges to implementation arose and when during the implementation process (Saldana et al., [<reflink idref="bib102" id="ref200">102</reflink>]). Ultimately, this information can facilitate frameworks' adoption and implementation with fidelity (i.e., intervention and implementation strategy).</p> <hd id="AN0175304773-20">Implementation Strategies for Multilevel Frameworks</hd> <p>Implementation scientists recommend that implementation strategies be selected and applied to address practice- and context-specific determinants (Baker et al., [<reflink idref="bib10" id="ref201">10</reflink>]; Powell et al., [<reflink idref="bib92" id="ref202">92</reflink>]). Yet, in practice, strategies are often mismatched to determinants (e.g., Davies et al., [<reflink idref="bib35" id="ref203">35</reflink>]). By delineating how implementation determinants interface with core features of multilevel service delivery frameworks, this study may provide a starting point for future research that evaluates the impact of determinants on implementation outcomes, and that seeks to identify/develop, operationally define, and test tailored implementation strategies (Powell et al., [<reflink idref="bib92" id="ref204">92</reflink>]). We highlighted implementation determinants at multiple contextual levels that appear to be salient for multilevel service delivery frameworks. Our findings indicate that the most consequential barriers and facilitators for each framework component may vary. As such, it is likely that the strategic integration of multiple implementation strategies ("baked in" and supplemental), tailored to the multilevel determinants most salient for each framework component and context (Powell et al., [<reflink idref="bib92" id="ref205">92</reflink>]), is needed to advance the high-quality implementation of multilevel service delivery frameworks.</p> <p>Although we identified numerous implementation strategies that are baked in as core components of multilevel service delivery frameworks, additional research is needed to systematically identify and subsequently specify how integrated strategies manifest across service models. Proctor et al. ([<reflink idref="bib95" id="ref206">95</reflink>]) developed guidelines to promote the precise description of implementation strategies applied in research or practice, including specification of strategies' actor, action, action target, temporality, dose, implementation outcome addressed, and justification. For example, whereas changing record systems via the development of clinical/data information systems is a key implementation strategy for CoCM and PBIS, precise descriptions of how these data information systems are developed and used—including the specific activities that are involved, in what sequence, performed by whom, and with what purpose—are needed. Specification of built in strategies as part of clinical (e.g., Moore et al., [<reflink idref="bib83" id="ref207">83</reflink>]; Rudd et al., [<reflink idref="bib99" id="ref208">99</reflink>]) and implementation research (Proctor et al., [<reflink idref="bib95" id="ref209">95</reflink>]), and tracking their use in implementation evaluations (Powell et al., [<reflink idref="bib93" id="ref210">93</reflink>]), is critical to guide the tailoring of implementation strategies for multilevel frameworks to a given service setting. Clearly identifying and specifying strategies that are built into a specific service framework may further support generalization and knowledge sharing across service contexts.</p> <p>The implementation determinants that we described in this paper for each core component may serve as a preliminary guidepost in identifying strategies, in addition to those that are baked in, to support multilevel service delivery framework implementation. Although a comprehensive review of strategies that might benefit multilevel frameworks is beyond our current scope, our work suggests that coordination of implementation strategies targeting multiple contextual levels (i.e., intervention, individuals, inner and outer setting) is needed. Given that systems-level organizational change is foundational to multilevel service delivery frameworks, organization-level implementation strategies appear central to multifaceted implementation strategy design. For example, researchers and practitioners may draw upon the Leadership and Organizational Change for Implementation (LOCI) model (Aarons et al., [<reflink idref="bib1" id="ref211">1</reflink>]) to improve implementation leadership and develop organizational strategies to support multilevel framework implementation (see Brookman-Frazee & Stahmer, [<reflink idref="bib19" id="ref212">19</reflink>] for an example adaptation study). The LOCI model includes six components (e.g., assessment, leadership training, coaching, organizational strategy development) and has evidence for its feasibility and acceptability among mental health team members (Aarons et al., [<reflink idref="bib1" id="ref213">1</reflink>]). Financial strategies, tailored to the funding context for each setting, that facilitate sustained investment in service delivery framework implementation also appear critical. For example, whereas time-limited grant funds may support initial adoption of PBIS, state-level PBIS implementers recommended identifying reliable (i.e., recurring) and diverse funding streams to support scale-up and sustainability (Gage et al., [<reflink idref="bib44" id="ref214">44</reflink>]). Federal Title IV, Part A funds from the Every Student Succeeds Act of 2015 may be used for this purpose (von Ravensberg, [<reflink idref="bib126" id="ref215">126</reflink>]). However, prior research on complex innovations in primary care settings suggests that strategies targeting individual professionals are more often used and evaluated than those targeting organizations or context (Lau et al., [<reflink idref="bib67" id="ref216">67</reflink>]). Thus, it is imperative that implementation researchers continue to broaden their focus to specify (Proctor et al., [<reflink idref="bib95" id="ref217">95</reflink>]) and test implementation strategies that target aspects of the inner and outer organizational contexts.</p> <hd id="AN0175304773-21">Limitations and Future Directions</hd> <p>Despite the potential of this conceptual paper to inform multilevel service delivery framework implementation, we acknowledge its limitations. Principally, we focused on three exemplar multilevel service delivery frameworks and did not complete a systematic review of the literature to identify core components and implementation determinants across the three exemplar frameworks included in this study. Our intention was to outline a conceptual foundation of commonalities and variation among framework components and implementation considerations that can span child mental health service settings. However, our focus on three exemplar frameworks (i.e., CoCM, PBIS, SoC), including one from each of three prominent youth mental health service settings, may have influenced the core components we identified as well as related implementation determinants. For example, it is possible that we may have identified additional or different core components had we focused on alternative frameworks (e.g., Interconnected Systems Framework) and/or service settings (e.g., juvenile justice). Similarly, relevant implementation determinants may also vary between frameworks and settings, such that frameworks that are emerging or less widely adopted may present with unique implementation challenges. Thus, our findings may not generalize to other multilevel frameworks or other settings. Research that expands upon this study to include additional frameworks, including those that are less widespread in their adoption and implementation, is needed. In addition, the foundation we articulated here will benefit from further refinement and elaboration through systematic reviews and coding of the bodies of literature for the exemplar multilevel service delivery frameworks; the components identified in this study should be considered preliminary until that work is completed. Reviews that articulate core components of each service delivery framework and the factors affecting their implementation (see Wood et al., [<reflink idref="bib132" id="ref218">132</reflink>] for an example), and that are able to bridge across service settings to improve interdisciplinary implementation, are needed.</p> <p>We highlighted prominent implementation determinants for each multilevel framework core component and coded these determinants using the CFIR. However, we did not engage in a systematic review process to identify these determinants. Determinants were identified from a range of articles and originated from study data or were extracted from manuscript text (e.g., results, discussion sections). Future systematic reviews of the literature that identify determinants from empirical evaluations of implementation barriers, facilitators, and outcomes are needed to more conclusively indicate which determinants are consequential in affecting implementation outcomes. Additionally, although the CFIR is effective for identifying and coding determinants across settings, we may have identified different determinants or augmented our discussion of relevant determinants had we used another framework (e.g., Theoretical Domains Framework; Cane et al., [<reflink idref="bib20" id="ref219">20</reflink>]). For example, the interplay between inner and outer setting factors that affects the cross-linking of service sectors as part of multilevel service delivery framework implementation can be represented by bridging factors in the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework (Lengnick-Hall et al., [<reflink idref="bib69" id="ref220">69</reflink>]). Implementation of this component of multilevel frameworks may be further supported by additional research that investigates potential bridging factors.</p> <p>Finally, identifying implementation determinants related to the core components of each framework was also challenging due to variability in terminology and focus in the relevant bodies of literature. Despite an acknowledged implementation gap across child mental health service settings, the application of implementation science and its associated terminology has progressed at variable rates across settings. For example, multiple scholars have acknowledged the underutilization of implementation science to date in schools compared to other settings (Forman et al., [<reflink idref="bib42" id="ref221">42</reflink>]; Sanetti & Collier-Meek, [<reflink idref="bib104" id="ref222">104</reflink>]). Though discussed in relation to overall framework implementation, implementation determinants were rarely discussed in alignment with core features of PBIS or SoC nor connected to implementation frameworks. This discrepancy highlights the importance of efforts to increase implementation science literacy outside of traditional health care settings (e.g., Sanetti & Collier-Meek, [<reflink idref="bib104" id="ref223">104</reflink>]), as well as the urgent opportunity to leverage interdisciplinary knowledge and collaboration to advance multilevel service delivery framework implementation. Researchers of PBIS, for example, might draw from the CoCM literature to inform and advance their own work related to PBIS implementation.</p> <hd id="AN0175304773-22">Conclusion</hd> <p>The goals of this paper were to integrate the diverse literatures on widely used multilevel service delivery frameworks in child mental health services as well as to serve as a starting point for cross-sector and interdisciplinary scholarship and practice by specifying common core components and multilevel implementation determinants that interact with each component. Our hope is that this foundation can inform efforts to design and evaluate implementation strategies capable of addressing the multitude of barriers to multilevel service delivery framework implementation and which are tailored to these frameworks' core features and context. Improving the high-quality implementation of multilevel service delivery frameworks may ultimately support increased access to high-quality mental health care for youth across a wide range of settings.</p> <hd id="AN0175304773-23">Acknowledgements</hd> <p>The authors would like to thank Natalie Romer for her contributions to the conceptualization of this manuscript's topic and scope. We would also like to thank the organizers and our colleagues at the 9th Annual School Mental Health Research Summit, hosted by the National Center for School Mental Health (NCSMH) and the School Mental Health Assessment and Research Training (SMART) Center, for bringing our authorship team together.</p> <hd id="AN0175304773-24">Author Contributions</hd> <p>All authors contributed to the conceptualization and refinement of the manuscript's topic and scope. SAM reviewed the literature and drafted preliminary text regarding core features of collaborative care, with feedback and review from ARL. JMC reviewed the literature and drafted preliminary text regarding core features of positive behavioral interventions and supports, with review and feedback from SAM. JM reviewed the literature and drafted preliminary text regarding core features of system of care, with feedback and review from SAM. SAM, JMC, JM, ARL contributed to the delineation of core components spanning each framework and to the identification of implementation determinants for respective frameworks from the literature. SAM led identification of determinants for two frameworks, with review and feedback from ARL and JM. JMC led identification for one framework, with review and feedback from SAM. SAM led coding of implementation determinants, with review and feedback from JMC, ARL, and JM. The first draft of the manuscript was written by SAM; JMC contributed to the introduction and discussion. SAM, JMC, JM, ARL revised and/or commented on previous versions of the manuscript and have read and approved the final manuscript.</p> <hd id="AN0175304773-25">Funding</hd> <p>Preparation of this article was supported in part by the National Institute of Mental Health through Grants T32MH109436 and R01MH119148. The opinions expressed are those of the authors and do not represent the views of the National Institutes of Health. The University of California libraries provided partial support for the open access publishing of this article. </p> <hd id="AN0175304773-26">Declarations</hd> <p></p> <hd id="AN0175304773-27">Competing interest</hd> <p>The authors have no competing interests to declare that are relevant to the content of this article.</p> <hd id="AN0175304773-28">Ethical Approval</hd> <p>This study did not involve human subjects research, thus no ethical approval was required.</p> <hd id="AN0175304773-29">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0175304773-30"> <title> References </title> <blist> <bibl id="bib1" idref="ref1" type="bt">1</bibl> <bibtext> Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science. 2015; 10: 11. 10.1186/s13012-014-0192-y. 25592163. 4310135</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> AIMS Center. (2014). Patient-centered integrated behavioral health care principles & tasks checklist.https://aims.uw.edu/sites/default/files/CollaborativeCarePrinciplesAndComponents_2014-12-23.pdf</bibtext> </blist> <blist> <bibl id="bib3" idref="ref5" type="bt">3</bibl> <bibtext> AIMS Center. (2021). Evidence base for COCM.https://aims.uw.edu/collaborative-care/evidence-base-cocm</bibtext> </blist> <blist> <bibl id="bib4" idref="ref6" type="bt">4</bibl> <bibtext> AIMS Center. (2023). Collaborative Care.https://aims.uw.edu/collaborative-care</bibtext> </blist> <blist> <bibl id="bib5" idref="ref7" type="bt">5</bibl> <bibtext> Alegría, M, Green, J. G, McLaughlin, K, & Loder, S. (2015). Disparities in Child and Adolescent Mental Health and Mental Health Services in the US. William T. Grant Foundation.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref8" type="bt">6</bibl> <bibtext> Algozzine, B, Barrett, S, Eber, L, George, H, Horner, R, Lewis, T, Putnam, B, Swain-Bradway, J, McIntosh, K, & Sugai, G. (2019). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. <ulink href="http://www.pbis.org">www.pbis.org</ulink>.</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> Asarnow JR, Jaycox LH, Duan N, LaBorde AP, Rea MM, Murray P, Anderson M, Landon C, Tang L, Wells KB. Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. JAMA. 2005; 293; 3: 311-319. 1:CAS:528:DC%2BD2MXmslarug%3D%3D. 10.1001/jama.293.3.311. 15657324</bibtext> </blist> <blist> <bibl id="bib8" idref="ref50" type="bt">8</bibl> <bibtext> Asarnow JR, Rozenmna M, Wiblin J, Zelter L. Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: A meta-analysis. JAMA Pediatrics. 2015; 169; 10: 929-937. 10.1001/jamapediatrics.2015.114. 26259143</bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext> Archer J, Bower P, Gilbody S, Lovell K, Richards D, Gask L, Dickens C, Coventry P. Collaborative care for depression and anxiety problems. Cochrane Database of Systematic Reviews. 2012; 10: CD006525. 10.1002/14651858.CD006525.pub2. 23076925</bibtext> </blist> <blist> <bibtext> Baker R, Comosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N, Wensing M, Fiander M, Eccles MP, Godycki-Cwirko M, van Lieshout J, Jäger C. Tailored interventions to address determinants of practice. Cochrane Database Systematic Review. 2015; 4: CD005470. 10.1002/14651858.CD005470.pub3</bibtext> </blist> <blist> <bibtext> Bao Y, Druss BG, Jung H-Y, Chan Y-F, Unützer J. Unpacking collaborative care for depression: Examining two essential tasks for implementation. Psychiatric Services. 2016; 67: 418-424. 10.1176/appi.ps.201400577. 26567934</bibtext> </blist> <blist> <bibtext> Behar L, Hydaker WM. Defining community readiness for the implementation of a system of care. Administration and Policy in Mental Health. 2009; 36: 381. 10.1007/s10488-009-0227-x. 19526338</bibtext> </blist> <blist> <bibtext> Bickman L, Lyon AR, Wolpert M. Achieving precision mental health through effective assessment, monitoring, and feedback processes. Administration and Policy in Mental Health and Mental Health Services Research. 2016; 43: 271-276. 10.1007/s10488-016-0718-5. 26887937</bibtext> </blist> <blist> <bibtext> Blase, K, & Fixsen, D. (2013). Core intervention components: Identifying and operationalizing what makes programs work. ASPE Research Brief. U.S. Department of Health and Human Services.</bibtext> </blist> <blist> <bibtext> Bradshaw CP, Koth CW, Thornton LA, Leaf PJ. Altering school climate through school-wide positive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science. 2009; 10: 100. 10.1007/s11121-008-0114-9. 19011963</bibtext> </blist> <blist> <bibtext> Bradshaw CP, Mitchell MM, Leaf PJ. Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes. Journal of Positive Behavior Interventions. 2010; 12; 3: 133-148. 10.1177/1098300709334798</bibtext> </blist> <blist> <bibtext> Bradshaw CP, Pas ET, Bloom J, Barrett S, Hershfeldt P, Alexander A, McKenna M, Chafin AE, Leaf PJ. A state-wide partnership to promote safe and supportive schools: The PBIS Maryland initiative. Administration and Policy in Mental Health. 2012; 39: 225-237. 10.1007/s10488-011-0384-6. 22246614</bibtext> </blist> <blist> <bibtext> Bronfenbrenner U, Ceci SJ. Nature-nurture reconceptualized in developmental perspective: A bioecological model. Psychological Review. 1994; 101; 4: 568-586. 1:STN:280:DyaK2M%2FnsF2mtw%3D%3D. 10.1037/0033-295x.101.4.568. 7984707</bibtext> </blist> <blist> <bibtext> Brookman-Frazee L, Stahmer AC. Effectiveness of a multi-level implementation strategy for ASD interventions: Study protocol for two linked cluster randomized trials. Implementation Science. 2018; 13: 66. 10.1186/s13012-018-0757-2. 29743090. 5944167</bibtext> </blist> <blist> <bibtext> Cane J, Oconnor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science. 2012; 7: 37. 10.1186/1748-5908-7-37. 22530986. 3483008</bibtext> </blist> <blist> <bibtext> Campo JV, Shafer S, Strohm J, Lucas A, Cassesse CG, Shaeffer D, Altman H. Pediatric behavioral health in primary care: A collaborative approach. Journal of the American Psychiatric Nurses Association. 2005; 11; 5: 276-282. 10.1177/1078390305282404</bibtext> </blist> <blist> <bibtext> Carlo AD, Unützer J, Ratzliff ADH, Cerimele JM. Financing for collaborative care: A narrative review. Current Treatment Options in Psychiatry. 2018; 5: 334-344. 10.1007/s40501-018-0150-4. 30083495. 6075691</bibtext> </blist> <blist> <bibtext> Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, and U.S. Department of Health and Human Services. (2015). The comprehensive community mental health services for children with serious emotional disturbances, evaluation findings—Report to Congress (2012–2013). U.S. Department of Health and Human Services.</bibtext> </blist> <blist> <bibtext> Center on Positive Behavioral Interventions and Supports (PBIS). Positive Behavioral Interventions and Supports (PBIS) implementation blueprint: Part 1—Foundations and supporting information. 2015; University of Oregon</bibtext> </blist> <blist> <bibtext> Center on Positive Behavioral Interventions and Supports (PBIS). (2023a). Positive Behavioral Interventions & Supports. https://<ulink href="http://www.pbis.org/pbis/what-is-pbis">www.pbis.org/pbis/what-is-pbis</ulink>.</bibtext> </blist> <blist> <bibtext> Center on Positive Behavioral Interventions and Supports (PBIS). (2023b). Data based decision making.https://<ulink href="http://www.pbis.org/topics/data-based-decision-making">www.pbis.org/topics/data-based-decision-making</ulink>.</bibtext> </blist> <blist> <bibtext> Chaparro, E. A, Horner, R, Algozzine, B, Daily, J, & Nese, R. N. T. (2022). How school teams use data to make effective decisions: Team-initiated problem solving (TIPS). Center on PBIS.</bibtext> </blist> <blist> <bibtext> Chitiyo M, Wheeler JJ. Challenges faced by school teachers in implementing positive behavior support in their school systems. Remedial & Special Education. 2009; 30; 1: 58-63. 10.1177/0741932508315049</bibtext> </blist> <blist> <bibtext> Consolidated Framework for Implementation Research (CFIR). (2023). 2009 CFIR Constructs. https://cfirguide.org/constructs-old/</bibtext> </blist> <blist> <bibtext> Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science. 2019; 20: 914-935. 10.1007/s11121-019-01017-1. 31152328. 8943907</bibtext> </blist> <blist> <bibtext> Costello EJ, He J, Sampson NA, Kessler RC, Merikangas KR. Services for adolescents with psychiatric disorders: 12-month data from the national comorbidity survey adolescent. Psychiatric Services. 2014; 65; 3: 359-366. 10.1176/appi.ps.201100518. 24233052</bibtext> </blist> <blist> <bibtext> Crable EL, Lengnick-Hall R, Stadnick NA, Moullin JC, Aarons GA. Where is "policy" in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example. Implementation Science. 2022; 17: 80. 10.1186/s13012-022-01256-x. 36503520. 9742035</bibtext> </blist> <blist> <bibtext> Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science. 2009; 4: 50. 10.1186/1748-5908-4-50. 19664226. 2736161</bibtext> </blist> <blist> <bibtext> Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science. 2022; 17: 75. 10.1186/s13012-022-01245-0. 36309746. 9617234</bibtext> </blist> <blist> <bibtext> Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implementation Science. 2010; 5; 1: 14. 10.1186/1748-5908-5-14. 20181130. 2832624</bibtext> </blist> <blist> <bibtext> Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008; 41; 3–4: 327-350. 10.1007/s10464-008-9165-0. 18322790</bibtext> </blist> <blist> <bibtext> Duong MT, Bruns EJ, Lee K, Cox S, Coifman J, Mayworm A, Lyon AR. Rates of mental health service utilization by children and adolescents in schools and other common service settings: A systematic review and meta-analysis. Administration and Policy in Mental Health and Mental Health Services Research. 2021; 48; 3: 420-439. 10.1007/s10488-020-01080-9. 32940884</bibtext> </blist> <blist> <bibtext> Eiraldi R, McCurdy B, Schwartz B, Wolk CB, Abraham M, Jawad AF, Nastasi BK, Mautone JA. Pilot study for the fidelity, acceptability, and effectiveness of a PBIS program plus mental health supports in under-resourced urban schools. Psychology in the Schools. 2019; 56; 8: 1230-1245. 10.1002/pits.22272. 33981121. 8112231</bibtext> </blist> <blist> <bibtext> Fabiano GA, Evans SW. Introduction to the special issue of School Mental Health on best practices in effective multi-tiered intervention frameworks. School Mental Health. 2019; 11; 1: 1-3. 10.1007/s12310-018-9283-2</bibtext> </blist> <blist> <bibtext> Flannery KB, Sugai G, Anderson CM. Schoolwide positive behavior support in high school: Early lessons learned. Journal of Positive Behavior Interventions. 2009; 11; 3: 177-185. 10.1177/1098300708316257</bibtext> </blist> <blist> <bibtext> Flannery KB, Fenning P, Kato MM, McIntosh K. Effects of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly. 2014; 29; 2: 111-124. 1:STN:280:DC%2BC2c7is1aqsw%3D%3D. 10.1037/spq0000039. 24188290</bibtext> </blist> <blist> <bibtext> Forman SG, Shapiro ES, Codding RS, Gonzales JE, Reddy LA, Rosenfield SA, Sanetti LMH, Stoiber KC. Implementation science and school psychology. School Psychology Quarterly. 2013; 28: 77-100. 10.1037/spq0000019. 23586516</bibtext> </blist> <blist> <bibtext> Fox RA, Leif ES, Moore DW, Furlonger B, Anderson A, Sharma U. A systematic review of the facilitators and barriers to the sustained implementation of school-wide positive behavioral interventions and supports. Education and Treatment of Children. 2022; 45: 105-126. 10.1007/s43494-021-00056-0</bibtext> </blist> <blist> <bibtext> Gage NA, Freeman J, Horner RH, Sugai G, Lewis TJ, Strickland-Cohen K. Funding support for school-wide positive behavioral interventions and support implementation and scale-up: A descriptive study of nine states. Journal of Disability Policy Studies. 2014; 25; 3: 164-174. 10.1177/1044207313488943</bibtext> </blist> <blist> <bibtext> George HP, Cox KE, Minch D, Sandomierski T. District practices associated with successful SWPBIS implementation. Behavioral Disorders. 2018; 43; 3: 393-406. 10.1177/0198742917753612</bibtext> </blist> <blist> <bibtext> Goforth AN, Farmer RL, Kim SY, Naser SC, Lockwood AB, Affrunti NW. Status of school psychology in 2020: Part 1, demographics of the NASP membership survey. NASP Research Reports. 2021; 5; 2: 1-17</bibtext> </blist> <blist> <bibtext> Goodrich DE, Kilbourne AM, Nord KM, Bauer MS. Mental health collaborative care and its role in primary care settings. Current Psychiatry Reports. 2013; 15: 383. 10.1007/s11920-013-0383-2. 23881714. 3759986</bibtext> </blist> <blist> <bibtext> Graaf G, Accomazzo S, Matthews K, Mendenhall A, Grube W. Evidence based practice in systems of care for children with complex mental health needs. Journal of Evidence-Based Social Work. 2021; 18; 4: 394-412. 10.1080/26408066.2021.1891172. 33827388</bibtext> </blist> <blist> <bibtext> Health Resources and Services Administration (HRSA). (2022). Designated health professional shortage areas statistics: Third quarter of fiscal year 2022. Bureau of Health Workforce, HRSA, U.S. Department of Health & Human Services</bibtext> </blist> <blist> <bibtext> Heflinger CA. Implementing a system of care: Findings from the Fort Bragg evaluation project. The Journal of Mental Health Administration. 1996; 23; 1: 16-29. 1:STN:280:DyaK287msl2ksA%3D%3D. 10.1007/bf02518640. 10154316</bibtext> </blist> <blist> <bibtext> Hodgkinson S, Godoy L, Beers LS, Lewin A. Improving mental health access for low-income children and families in the primary care setting. Pediatrics. 2017; 139; 1: 0151175. 10.1542/peds.2015-1175</bibtext> </blist> <blist> <bibtext> Hoover S, Bostic J. Schools as a vital component of the child and adolescent mental health system. Psychiatric Services. 2021; 72; 1: 37-48. 10.1176/appi.ps.201900575. 33138711</bibtext> </blist> <blist> <bibtext> Horner RH, Sugai G. School-wide PBIS: An example of applied behavior analysis implemented at a scale of social importance. Behavior Analysis in Practice. 2015; 8; 1: 80-85. 10.1007/s40617-015-0045-4. 27703887. 5048248</bibtext> </blist> <blist> <bibtext> Horner RH, Sugai G, Smolkowski K, Eber L, Nakasato J, Todd AW, Esperanza J. A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions. 2009; 11; 3: 133-144. 10.1177/1098300709332067</bibtext> </blist> <blist> <bibtext> Houtrow AJ, Larson K, Olson LM, Newacheck PW, Halfon N. Changing trends of childhood disability, 2001–2011. Pediatrics. 2014; 134; 3: 530-538. 10.1542/peds.2014-0594. 25136051. 4144000</bibtext> </blist> <blist> <bibtext> Ikeda MJ, Tilly WD, Stumme J, Volmer L, Allison R. Agency wide implementation of problem solving consultation: Foundations, current implementation, and future directions. School Psychology Quarterly. 1996; 11: 228-243. 10.1037/h0088931</bibtext> </blist> <blist> <bibtext> Jimerson SR, Burns MK, VanDerHeyden AM. The handbook of response to intervention: Science and practice of multi-tiered systems of support. 20162; Springer Science</bibtext> </blist> <blist> <bibtext> Katon W, Unützer J, Wells K, Jones L. Collaborative depression care: History, evolution and ways to enhance dissemination and sustainability. General Hospital Psychiatry. 2010; 32: 456-464. 10.1016/j.genhosppsych.2010.04.001. 20851265</bibtext> </blist> <blist> <bibtext> Kelm JL, McIntosh K, Cooley S. Effects of implementing school-wide positive behavioural interventions and supports on problem behaviour and academic achievement in a Canadian elementary school. Canadian Journal of School Psychology. 2014; 29; 3: 195-212. 10.1177/0829573514540266</bibtext> </blist> <blist> <bibtext> Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implementation Science. 2020; 11: 72. 10.1186/s13012-016-0437-z</bibtext> </blist> <blist> <bibtext> Kincaid D, Childs K, Blasé KA, Wallace F. Identifying barriers and facilitators in implementing schoolwide positive behavior support. Journal of Positive Behavior Interventions. 2007; 9; 3: 174-184. 10.1177/10983007070090030501</bibtext> </blist> <blist> <bibtext> Kolko DJ, Campo J, Kelleher K, Cheng Y. Improving access to care and clinical outcome for pediatric behavioral problems: A randomized trial of a nurse-administered intervention in primary care. Journal of Developmental & Behavioral Pediatrics. 2010; 31; 5: 393-404. 10.1097/DBP.0b013e3181dff307</bibtext> </blist> <blist> <bibtext> Kolko DJ, Campo J, Kilbourne AM, Hart J, Sakolsky D, Wisniewski S. Collaborative care outcomes for pediatric behavioral health problems: A cluster randomized trial. Pediatrics. 2014; 133; 4: e981-e992. 10.1542/peds.2013-2516. 24664093. 3966503</bibtext> </blist> <blist> <bibtext> Kolko DJ, Perrin E. The integration of behavioral health interventions in children's health care: Services, science, and suggestions. Journal of Clinical Child & Adolescent Psychology. 2014; 43; 2: 216-228. 10.1080/15374416.2013.862804</bibtext> </blist> <blist> <bibtext> Kumm S, Mathur SR, Cassavaugh M, Butts E. Using the PBIS framework to meet the mental health needs of youth in juvenile justice facilities. Remedial and Special Education. 2020; 41; 2: 80-87. 10.1177/0741932519880336</bibtext> </blist> <blist> <bibtext> Kutash K, Greenbaum PE, Wang W, Boothroyd RA, Friedman RM. Levels of system of care implementation: A national study. Journal of Behavioral Health Services & Research. 2011; 38; 3: 342-357. 10.1007/s11414-011-9242-2</bibtext> </blist> <blist> <bibtext> Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, Everitt H, Kennedy A, Qureshi N, Rogers A, Peacock R, Murray E. Achieving change in primary care—effectiveness of strategies for improving complex interventions: Systematic review of reviews. British Medical Journal Open. 2015; 5: e009993. 10.1136/bmjopen-2015-009993</bibtext> </blist> <blist> <bibtext> Lawson GM, McKenzie ME, Becker KD, Selby L, Hoover SA. The core components of evidence-based social emotional learning programs. Prevention Science. 2018; 20: 457-467. 10.1007/s11121-018-0953-y</bibtext> </blist> <blist> <bibtext> Lengnick-Hall R, Stadnick NA, Dickson KA, Moullin JC, Aarons GA. Forms and functions of bridging factors: Specifying the dynamic links between outer and inner contexts during implementation and sustainment. Implementation Science. 2021; 16: 34. 10.1186/s13012-021-01099-y. 33794956. 8015179</bibtext> </blist> <blist> <bibtext> Locke J, Kang-Yi CD, Pellechia M, Marcus S, Hadley T, Mandell DS. Ethnic disparities in school-based behavioral health service use for children with psychiatric disorders. Journal of School Health. 2017; 87; 1: 47-54. 10.1111/josh.12469. 27917490</bibtext> </blist> <blist> <bibtext> Lunn LM, Heflinger CA, Wang W, Greenbaum PE, Kutash K, Boothroyd RA, Friedman RM. Community characteristics and implementation factors associated with effective systems of care. Journal of Behavioral Health Services & Research. 2011; 38; 3: 327-341. 10.1007/s11414-011-9244-0</bibtext> </blist> <blist> <bibtext> Lyon AR, Whitaker K, French WP, Richardson LP, Wasse JK, McCauley E. Collaborative care in schools: Enhancing integration and impact in youth mental health. Advances in School Mental Health Promotion. 2016; 9; 3–4: 148-168. 10.1080/1754730X.2016.1215928. 28392832. 5383210</bibtext> </blist> <blist> <bibtext> McIntosh K, Bennett JL, Price K. Evaluation of social and academic effects of school-wide positive behaviour support in a Canadian school district. Exceptionality Education International. 2011; 21; 1: 46-60. 10.5206/eei.v21i1.7669</bibtext> </blist> <blist> <bibtext> McIntosh K, Goodman S. The Guilford practical intervention in the schools series Blending RTI and PBIS. 2016; Guilford Press</bibtext> </blist> <blist> <bibtext> McIntosh K, Kim J, Mercer SH, Strickland-Cohen MK, Horner RH. Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention. 2015; 40; 3: 184-191. 10.1177/1534508414556503</bibtext> </blist> <blist> <bibtext> McIntosh K, Mercer SH, Hume AE, Frank JL, Turri MG, Mathews S. Factors related to sustained implementation of schoolwide positive behavior support. Exceptional Children. 2013; 79; 3: 293-311. 10.3102/0013189X18776975</bibtext> </blist> <blist> <bibtext> Miller BD, Blau GM, Christopher OT, Jordan PE. Sustaining and expanding systems of care to provide mental health services for children, youth and families across America. American Journal of Community Psychology. 2012; 49: 566-579. 10.1007/s10464-012-9517-7. 22543719</bibtext> </blist> <blist> <bibtext> Mendenhall AN, Fraunholtz S. System of care development in children's mental health: Lessons learned from a process evaluation. Journal of Child and Family Studies. 2014; 23: 157-168. 10.1007/s10826-012-9701-2</bibtext> </blist> <blist> <bibtext> Merikangas KR, He JP, Burstein M, Swendsen J, Avenevoli S, Case B, Georgiades K, Heaton L, Swanson S, Olfson M. Service utilization for lifetime mental disorders in U.S. adolescents: Results of the national comorbidity survey-Adolescent supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry. 2011; 50; 1: 32-45. 10.1016/j.jaac.2010.10.006. 21156268</bibtext> </blist> <blist> <bibtext> Miles, P, Bruns, E.J, Osher, T.W, Walker, J.S, & National Wraparound Initiative Advisory Group (2006/2019). The Wraparound Process User's Guide: A Handbook for Families. National Wraparound Initiative, Research and Training Center on Family Support and Children's Mental Health, Portland State University</bibtext> </blist> <blist> <bibtext> Miron O, Yu K-H, Wilf-Miron R, Kohane IS. Suicide rates among adolescents and young adults in the United States, 2000–2017. Journal of the American Medical Association. 2019; 321; 23: 2362-2364. 10.1001/jama.2019.5054. 31211337</bibtext> </blist> <blist> <bibtext> Molloy LE, Moore JE, Trail J, Van Epps JJ, Hopfer S. Understanding real-world implementation quality and "active ingredients" of PBIS. Prevention Science. 2013; 14; 6: 593-605. 10.1007/s11121-012-0343-9. 23408283. 4032114</bibtext> </blist> <blist> <bibtext> Moore SA, Arnold KT, Beidas RS, Mendelson T. Specifying and reporting implementation strategies used in a school-based prevention efficacy trial. Implementation Research and Practice. 2021. 10.1177/26334895211047841. 34761223. 8577529</bibtext> </blist> <blist> <bibtext> National Association of School Psychologists. (2021). Shortages in school psychology: Challenges to meeting the growing needs of U.S. students and schools [Research summary].</bibtext> </blist> <blist> <bibtext> New Hampshire. (2018). §1915(i) State plan HCBS. https://<ulink href="http://www.dhhs.nh.gov/sites/g/files/ehbemt476/files/documents/2021-11/sp-3-1i.pdf">www.dhhs.nh.gov/sites/g/files/ehbemt476/files/documents/2021-11/sp-3-1i.pdf</ulink></bibtext> </blist> <blist> <bibtext> Nilsen P. Making sense of implementation theories, models, and frameworks. Implementation Science. 2015; 10: 53. 10.1186/s13012-015-0242-0. 25895742. 4406164</bibtext> </blist> <blist> <bibtext> Office of the Surgeon General. (2021). Protecting Youth Mental Health: The U.S. Surgeon General's Advisory. US Department of Health and Human Services.</bibtext> </blist> <blist> <bibtext> Overbeck G, Davidsen AS, Kousgaard MB. Enablers and barriers to implementing collaborative care for anxiety and depression: A systematic qualitative review. Implementation Science. 2016; 11: 165. 10.1186/s13012-016-0519-y. 28031028. 5192575</bibtext> </blist> <blist> <bibtext> Pas ET, Bradshaw CP. Examining the association between implementation and outcomes: State-wide scale-up of School-wide Positive Behavior Intervention and Supports. The Journal of Behavioral Health Services & Research. 2012; 39: 417-433. 10.1007/s11414-012-9290-2</bibtext> </blist> <blist> <bibtext> PBISApps. (2022). SWIS. https://<ulink href="http://www.pbisapps.org/products/swis">www.pbisapps.org/products/swis</ulink></bibtext> </blist> <blist> <bibtext> Pires S. Building systems of care: A primer for child welfare. 20102; Georgetown University Center for Child and Human Development</bibtext> </blist> <blist> <bibtext> Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, Mandell DS. Methods to improve the selection and tailoring of implementation strategies. Journal of Behavioral Health Services Research. 2017; 44; 2: 177-194. 10.1007/s11414-015-9475-6. 26289563</bibtext> </blist> <blist> <bibtext> Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, McHugh SM, Weiner BJ. Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health. 2019; 7: 3. 10.3389/fpubh.2019.00003. 30723713. 6350272</bibtext> </blist> <blist> <bibtext> Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015; 10: 21. 10.1186/s13012-015-0209-1. 25889199. 4328074</bibtext> </blist> <blist> <bibtext> Proctor EK, Powell BJ, McMillen JC. Implementation strategies: Recommendations for specifying and reporting. Implementation Science. 2013; 8: 139. 10.1186/1748-5908-8-139. 24289295. 3882890</bibtext> </blist> <blist> <bibtext> Richardson LP, Ludman E, McCauley E, Lindenbaum J, Larison C, Zhou C, Clarke G, Brent D, Katon W. Collaborative care for adolescents with depression in primary care: A randomized clinical trial. JAMA. 2014; 312; 8: 809-816. 1:CAS:528:DC%2BC2cXhs12jtrrI. 10.1001/jama.2014.9259. 25157724. 4492537</bibtext> </blist> <blist> <bibtext> Richardson LP, McCarty CA, Radovic A, Suleiman AB. Research in integration of behavioral health for adolescents and young adults in primary care settings: A systematic review. Journal of Adolescent Health. 2017; 60; 3: 261-269. 10.1016/j.jadohealth.2016.11.013</bibtext> </blist> <blist> <bibtext> Ross SW, Endrulat NR, Horner RH. Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 2011; 13: 1-12</bibtext> </blist> <blist> <bibtext> Rudd BN, Davis M, Beidas RS. Integrating implementation science in clinical research to maximize public health impact: A call for the reporting of implementation strategy use with implementation outcomes in clinical research. Implementation Science. 2020; 15: 103. 10.1186/s13012-020-01060-5. 33239097. 7690013</bibtext> </blist> <blist> <bibtext> Saldana L, Chamberlain P, Wang W, Brown H. Predicting program start-up using the stages of implementation measure. Administration and Policy in Mental Health and Mental Health Services Research. 2011; 39: 419-425. 10.1007/s10488-011-0363-y</bibtext> </blist> <blist> <bibtext> Saldana L. The stages of implementation completion for evidence-based practice: Protocol for a mixed methods study. Implementation Science. 2014; 9: 43. 3264269. 10.1186/1748-5908-9-43. 24708893. 4234147</bibtext> </blist> <blist> <bibtext> Saldana L, Bennett I, Powers D, Vredevoogd M, Grover T, Schaper H, Campbell M. Scaling implementation of collaborative care for depression: Adaptation of the Stages of Implementation Completion (SIC). Administration and Policy in Mental Health. 2020; 47: 188-196. 10.1007/s10488-019-00944-z. 31197625. 6908762</bibtext> </blist> <blist> <bibtext> Sanchez K, Thompson S, Alexander L. Current strategies and barriers to integrated health care: A survey of publicly funded providers in Texas. General Hospital Psychiatry. 2010; 32; 1: 26-32. 10.1016/j.genhosppsych.2009.10.007. 20114125</bibtext> </blist> <blist> <bibtext> Sanetti L, Collier-Meek M. Increasing implementation science literacy to address the research-to-practice gap in school psychology. Journal of School Psychology. 2019; 76: 33-47. 10.1016/j.jsp.2019.07.008</bibtext> </blist> <blist> <bibtext> Sanetti LMH, Cook BG, Cook L. Treatment fidelity: What it is and why it matters. Learning Disabilities Research & Practice. 2021; 36; 1: 5-11. 10.1111/ldrp.12238</bibtext> </blist> <blist> <bibtext> Schultz BK, Al-Hammori D, Mirabelli K, Gaither L. Mental health services in North Carolina's public schools. North Carolina Medical Journal. 2020; 81; 2: 111-115. 10.18043/ncm.81.2.111. 32132253</bibtext> </blist> <blist> <bibtext> Schurer J, Bruns E, Hensley S, Paragoris R. Wraparound Implementation and Quality Practice Standards. 2016; National Technical Assistance Network for Children's Behavioral Health</bibtext> </blist> <blist> <bibtext> Simon AE, Pastor PN, Reuben CA, Huang LN, Goldstrom ID. Use of mental health services by children ages six to 11 with emotional or behavioral difficulties. Psychiatric Services. 2015; 66; 9: 930-937. 10.1176/appi.ps.201400342. 25975889</bibtext> </blist> <blist> <bibtext> Smith JD, Fu E, Rado J, Rosenthal LJ, Carroll AJ, Atlas JA, Carlo AD, Burnett-Zeigler I, Jordan N, Brown CH, Csernansky J. Collaborative care for depression management in primary care: A randomized roll-out trial using type 2 hybrid effectiveness implementation design. Contemporary Clinical Trials Communications. 2021; 23: 100823. 10.1016/j.conctc.2021.100823. 34401595. 8350002</bibtext> </blist> <blist> <bibtext> Solberg LI, Crain AL, Jaeckels N, Ohnsorg KA, Margolis KL, Beck A, Whitebird RR, Rossom RC, Crabtree BF, Van de Ven AH. The DIAMOND initiative: Implementing collaborative care for depression in 75 primary care clinics. Implementation Science. 2013; 8: 135. 10.1186/1748-5908-8-135. 24238225. 3842646</bibtext> </blist> <blist> <bibtext> Splett JW, Perales K, Halliday-Boykins CA, Gilchrest CE, Gibson N, Weist MD. Best practices for teaming and collaboration in the interconnected systems framework. Journal of Applied School Psychology. 2017; 33; 4: 347-368. 10.1080/15377903.2017.1328625</bibtext> </blist> <blist> <bibtext> Stroul, B. A. (2013). Lessons learned for expanding systems of care: Analysis of the system of care expansion planning grant program. Technical Assistance Partnership for Child and Family Mental Health.</bibtext> </blist> <blist> <bibtext> Stroul BA, Blau GM, Larson J. The evolution of the system of care approach for children, youth, and young adults with mental health conditions and their families. 2021; School of Social Work, University of Maryland</bibtext> </blist> <blist> <bibtext> Stroul, B, Blau, G, & Friedman, R. (2010). Updating the system of care concept and philosophy. Washington, DC: Georgetown University Center for Child and Human Development, National Technical Assistance Center for Children's Mental Health.</bibtext> </blist> <blist> <bibtext> Stroul B, Blau G, Sondheimer DStroul B, Blau G. Systems of care: A strategy to transform children's mental health care. The system of care handbook: Transforming mental health services for children, youth and families. 2008; Paul H. Brookes Publishing Co: 3-24</bibtext> </blist> <blist> <bibtext> Stroul, B, Dodge, J, Goldman, S, Rider, F, & Friedman, R. (2015). Toolkit for Expanding the System of Care Approach. Georgetown University Center for Child and Human Development, National Technical Assistance Center for Children's Mental Health.</bibtext> </blist> <blist> <bibtext> Stroul BA, Friedman RM. A system of care for children and youth with severe emotional disturbances. 1986Revised; Georgetown University Child Development Center</bibtext> </blist> <blist> <bibtext> Stroul BA, Manteuffel BA. The sustainability of systems of care for children's mental health: Lessons learned. Journal of Behavioral Health Services & Research. 2007; 34; 3: 237-259. 10.1007/s11414-007-9065-3</bibtext> </blist> <blist> <bibtext> Substance Abuse and Mental Health Services Administration (SAMHSA). (2020). National guidelines for behavioral health crisis care: Best practice toolkit. Author. https://<ulink href="http://www.samhsa.gov/sites/default/files/national-guidelines-for-behavioral-health-crisis-care-02242020.pdf">www.samhsa.gov/sites/default/files/national-guidelines-for-behavioral-health-crisis-care-02242020.pdf</ulink></bibtext> </blist> <blist> <bibtext> Suter JC, Bruns EJ. Effectiveness of the wraparound process for children with emotional and behavioral disorders: A meta-analysis. Clinical Child and Family Psychology Review. 2009; 12: 336-351. 10.1007/s10567-009-0059-y. 19834802</bibtext> </blist> <blist> <bibtext> The White House. (2022). Fact sheet: Biden-Harris administration announces two new actions to address youth mental health crisis. https://<ulink href="http://www.whitehouse.gov/briefing-room/statements-releases/2022/07/29/fact-sheet-biden-harris-administration-announces-two-new-actions-to-address-youth-mental-health-crisis/">www.whitehouse.gov/briefing-room/statements-releases/2022/07/29/fact-sheet-biden-harris-administration-announces-two-new-actions-to-address-youth-mental-health-crisis/</ulink></bibtext> </blist> <blist> <bibtext> Thota AB, Sipe TA, Brard GJ, Zometa CS, Hahn RA, McKnight-Eily LR, Chapman DP, Abraido-Lanza AF, Pearson JL, Anderson CW, Gelenberg AJ, Hennessy KD, Duffy FF, Vernon-Smiley ME, Nease DE Jr, Williams S Community Preventive Services Task Force. Collaborative care to improve the management of depressive disorders: A community guide systematic review and meta-analysis. American Journal of Preventive Medicine. 2012; 42; 5: 525-538. 10.1016/j.amepre.2012.01.019. 22516495</bibtext> </blist> <blist> <bibtext> Unützer, J, Harbin, H, Schoenbaum, M, & Druss, B. (2013). The collaborative care model: An approach for integrating physical and mental health care in Medicaid health homes. Health Home, Information Resource Center.</bibtext> </blist> <blist> <bibtext> Vinson NB, Brannan AM, Baughman LN, Wilce M, Gawron T. The system-of-care model: Implementation in twenty-seven communities. Journal of Emotional and Behavioral Disorders. 2001; 9; 1: 30-42. 10.1177/106342660100900104</bibtext> </blist> <blist> <bibtext> von Ravensberg, H. (2020). Funding PBIS implementation through ESSA Title IV, Part A funds. SSRN. https://doi.org/10.2139/ssrn.3526527</bibtext> </blist> <blist> <bibtext> Walker, J. S, Bruns, E. J, & The National Wraparound Initiative Advisory Group. (2008). Phases and activities of the wraparound process. In E. J. Bruns & J. S. Walker (Eds.), The resource guide to wraparound. National Wraparound Initiative, Research and Training Center for Family Support and Children's Mental Health.</bibtext> </blist> <blist> <bibtext> Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: Results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science. 2015; 10: 109. 10.1186/s13012-015-0295-0. 26249843. 4527340</bibtext> </blist> <blist> <bibtext> Whitebird RR, Solberg LI, Jaeckels NA, Piertruszewski PB, Hadzic S, Unützer J, Ohnsorg KA, Rosson RC, Beck A, Joslyn KE, Rubenstein LV. Effective implementation of collaborative care for depression: What is needed?. The American Journal of Managed Care. 2014; 20; 9: 699-707. 10.3121/cmr.2014.1250.ps1-22. 25365745. 4270471</bibtext> </blist> <blist> <bibtext> Wright DR, Haaland WL, Ludman E, McCauley E, Lindenbaum J, Richardson LP. The costs and cost-effectiveness of collaborative care for adolescents with depression in primary care settings: A randomized clinical trial. JAMA Pediatrics. 2016; 170; 11: 1048-1054. 10.1001/jamapediatrics.2016.1721. 27654449</bibtext> </blist> <blist> <bibtext> Wolk CB, Last BS, Livsey C, Oquendo MA, Press MJ, Mandell DS, Ingram E, Futterer AC, Kinkle GP, Oslin DW. Addressing common challenges in the implementation of collaborative care for mental health: The Penn integrated care program. Annals of Family Medicine. 2021; 19; 2: 148-156. 10.1370/afm.2651. 33685876. 7939709</bibtext> </blist> <blist> <bibtext> Woltman E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: Systematic review and meta-analysis. American Journal of Psychiatry. 2012; 169; 8: 790-804. 10.1176/appi.ajp.2012.11111616</bibtext> </blist> <blist> <bibtext> Wood E, Ohlsen S, Rickets T. What are the barriers and facilitators to implementing collaborative care for depression? A systematic review. Journal of Affective Disorders. 2017; 214: 26-43. 10.1016/j.jad.2017.02.028. 28266319</bibtext> </blist> <blist> <bibtext> Yu H, Kolko DJ, Torres E. Collaborative mental health care for pediatric disorders in primary care: Does it reduce mental health care costs?. Families, Systems, & Health. 2017; 35; 1: 46-57. 10.1037/fsh0000251</bibtext> </blist> </ref> <ref id="AN0175304773-31"> <title> Footnotes </title> <blist> <bibtext> We use the general terms "provider" or "professional" to refer to individuals who are involved in the implementation of multilevel service delivery frameworks. However, we recognize that terminology used to describe partners involved in implementation may vary across frameworks, for example, including peer supporters and other implementers.</bibtext> </blist> </ref> <aug> <p>By Stephanie A. Moore; Jennifer McGrory Cooper; JoAnne Malloy and Aaron R. Lyon</p> <p>Reported by Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib55" firstref="ref9"></nolink> <nolink nlid="nl2" bibid="bib81" firstref="ref10"></nolink> <nolink nlid="nl3" bibid="bib31" firstref="ref11"></nolink> <nolink nlid="nl4" bibid="bib79" firstref="ref12"></nolink> <nolink nlid="nl5" bibid="bib109" firstref="ref13"></nolink> <nolink nlid="nl6" bibid="bib51" firstref="ref15"></nolink> <nolink nlid="nl7" bibid="bib70" firstref="ref16"></nolink> <nolink nlid="nl8" bibid="bib87" firstref="ref17"></nolink> <nolink nlid="nl9" bibid="bib37" firstref="ref18"></nolink> <nolink nlid="nl10" bibid="bib15" firstref="ref19"></nolink> <nolink nlid="nl11" bibid="bib38" firstref="ref20"></nolink> <nolink nlid="nl12" bibid="bib47" firstref="ref21"></nolink> <nolink nlid="nl13" bibid="bib97" firstref="ref22"></nolink> <nolink nlid="nl14" bibid="bib116" firstref="ref23"></nolink> <nolink nlid="nl15" bibid="bib114" firstref="ref24"></nolink> <nolink nlid="nl16" bibid="bib36" firstref="ref26"></nolink> <nolink nlid="nl17" bibid="bib25" firstref="ref28"></nolink> <nolink nlid="nl18" bibid="bib39" firstref="ref30"></nolink> <nolink nlid="nl19" bibid="bib88" firstref="ref31"></nolink> <nolink nlid="nl20" bibid="bib82" firstref="ref32"></nolink> <nolink nlid="nl21" bibid="bib101" firstref="ref33"></nolink> <nolink nlid="nl22" bibid="bib105" firstref="ref34"></nolink> <nolink nlid="nl23" bibid="bib68" firstref="ref36"></nolink> <nolink nlid="nl24" bibid="bib11" firstref="ref37"></nolink> <nolink nlid="nl25" bibid="bib132" firstref="ref38"></nolink> <nolink nlid="nl26" bibid="bib10" firstref="ref41"></nolink> <nolink nlid="nl27" bibid="bib67" firstref="ref42"></nolink> <nolink nlid="nl28" bibid="bib92" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib24" firstref="ref46"></nolink> <nolink nlid="nl30" bibid="bib43" firstref="ref47"></nolink> <nolink nlid="nl31" bibid="bib63" firstref="ref51"></nolink> <nolink nlid="nl32" bibid="bib48" firstref="ref52"></nolink> <nolink nlid="nl33" bibid="bib66" firstref="ref53"></nolink> <nolink nlid="nl34" bibid="bib17" firstref="ref55"></nolink> <nolink nlid="nl35" bibid="bib77" firstref="ref56"></nolink> <nolink nlid="nl36" bibid="bib89" firstref="ref57"></nolink> <nolink nlid="nl37" bibid="bib111" firstref="ref58"></nolink> <nolink nlid="nl38" bibid="bib53" firstref="ref59"></nolink> <nolink nlid="nl39" bibid="bib16" firstref="ref61"></nolink> <nolink nlid="nl40" bibid="bib73" firstref="ref62"></nolink> <nolink nlid="nl41" bibid="bib118" firstref="ref69"></nolink> <nolink nlid="nl42" bibid="bib23" firstref="ref70"></nolink> <nolink nlid="nl43" bibid="bib95" firstref="ref74"></nolink> <nolink nlid="nl44" bibid="bib128" firstref="ref75"></nolink> <nolink nlid="nl45" bibid="bib94" firstref="ref76"></nolink> <nolink nlid="nl46" bibid="bib30" firstref="ref77"></nolink> <nolink nlid="nl47" bibid="bib18" firstref="ref78"></nolink> <nolink nlid="nl48" bibid="bib115" firstref="ref80"></nolink> <nolink nlid="nl49" bibid="bib21" firstref="ref82"></nolink> <nolink nlid="nl50" bibid="bib72" firstref="ref86"></nolink> <nolink nlid="nl51" bibid="bib13" firstref="ref89"></nolink> <nolink nlid="nl52" bibid="bib58" firstref="ref97"></nolink> <nolink nlid="nl53" bibid="bib74" firstref="ref98"></nolink> <nolink nlid="nl54" bibid="bib80" firstref="ref99"></nolink> <nolink nlid="nl55" bibid="bib108" firstref="ref100"></nolink> <nolink nlid="nl56" bibid="bib127" firstref="ref101"></nolink> <nolink nlid="nl57" bibid="bib45" firstref="ref105"></nolink> <nolink nlid="nl58" bibid="bib90" firstref="ref111"></nolink> <nolink nlid="nl59" bibid="bib52" firstref="ref115"></nolink> <nolink nlid="nl60" bibid="bib107" firstref="ref116"></nolink> <nolink nlid="nl61" bibid="bib86" firstref="ref122"></nolink> <nolink nlid="nl62" bibid="bib33" firstref="ref123"></nolink> <nolink nlid="nl63" bibid="bib34" firstref="ref125"></nolink> <nolink nlid="nl64" bibid="bib60" firstref="ref127"></nolink> <nolink nlid="nl65" bibid="bib29" firstref="ref128"></nolink> <nolink nlid="nl66" bibid="bib93" firstref="ref130"></nolink> <nolink nlid="nl67" bibid="bib119" firstref="ref132"></nolink> <nolink nlid="nl68" bibid="bib113" firstref="ref133"></nolink> <nolink nlid="nl69" bibid="bib125" firstref="ref134"></nolink> <nolink nlid="nl70" bibid="bib130" firstref="ref137"></nolink> <nolink nlid="nl71" bibid="bib50" firstref="ref138"></nolink> <nolink nlid="nl72" bibid="bib78" firstref="ref139"></nolink> <nolink nlid="nl73" bibid="bib40" firstref="ref141"></nolink> <nolink nlid="nl74" bibid="bib26" firstref="ref145"></nolink> <nolink nlid="nl75" bibid="bib121" firstref="ref146"></nolink> <nolink nlid="nl76" bibid="bib12" firstref="ref148"></nolink> <nolink nlid="nl77" bibid="bib28" firstref="ref150"></nolink> <nolink nlid="nl78" bibid="bib76" firstref="ref152"></nolink> <nolink nlid="nl79" bibid="bib27" firstref="ref156"></nolink> <nolink nlid="nl80" bibid="bib46" firstref="ref161"></nolink> <nolink nlid="nl81" bibid="bib49" firstref="ref162"></nolink> <nolink nlid="nl82" bibid="bib84" firstref="ref164"></nolink> <nolink nlid="nl83" bibid="bib32" firstref="ref165"></nolink> <nolink nlid="nl84" bibid="bib61" firstref="ref177"></nolink> <nolink nlid="nl85" bibid="bib103" firstref="ref183"></nolink> <nolink nlid="nl86" bibid="bib122" firstref="ref193"></nolink> <nolink nlid="nl87" bibid="bib65" firstref="ref195"></nolink> <nolink nlid="nl88" bibid="bib120" firstref="ref197"></nolink> <nolink nlid="nl89" bibid="bib14" firstref="ref198"></nolink> <nolink nlid="nl90" bibid="bib100" firstref="ref199"></nolink> <nolink nlid="nl91" bibid="bib102" firstref="ref200"></nolink> <nolink nlid="nl92" bibid="bib35" firstref="ref203"></nolink> <nolink nlid="nl93" bibid="bib83" firstref="ref207"></nolink> <nolink nlid="nl94" bibid="bib99" firstref="ref208"></nolink> <nolink nlid="nl95" bibid="bib19" firstref="ref212"></nolink> <nolink nlid="nl96" bibid="bib44" firstref="ref214"></nolink> <nolink nlid="nl97" bibid="bib126" firstref="ref215"></nolink> <nolink nlid="nl98" bibid="bib20" firstref="ref219"></nolink> <nolink nlid="nl99" bibid="bib69" firstref="ref220"></nolink> <nolink nlid="nl100" bibid="bib42" firstref="ref221"></nolink> <nolink nlid="nl101" bibid="bib104" firstref="ref222"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1434362
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks across Child Mental Health Service Settings
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Stephanie+A%2E+Moore%22">Stephanie A. Moore</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1550-8344">0000-0003-1550-8344</externalLink>)<br /><searchLink fieldCode="AR" term="%22Jennifer+McGrory+Cooper%22">Jennifer McGrory Cooper</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0003-0509-1653">0000-0003-0509-1653</externalLink>)<br /><searchLink fieldCode="AR" term="%22JoAnne+Malloy%22">JoAnne Malloy</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7403-476X">0000-0002-7403-476X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Aaron+R%2E+Lyon%22">Aaron R. Lyon</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3657-5060">0000-0003-3657-5060</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Administration+and+Policy+in+Mental+Health+and+Mental+Health+Services+Research%22"><i>Administration and Policy in Mental Health and Mental Health Services Research</i></searchLink>. 2024 51(2):172-195.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Institute of Mental Health (NIMH) (DHHS/NIH)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: T32MH109436<br />R01MH119148
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Mental+Health%22">Mental Health</searchLink><br /><searchLink fieldCode="DE" term="%22Health+Services%22">Health Services</searchLink><br /><searchLink fieldCode="DE" term="%22Context+Effect%22">Context Effect</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Positive+Behavior+Supports%22">Positive Behavior Supports</searchLink><br /><searchLink fieldCode="DE" term="%22Primary+Health+Care%22">Primary Health Care</searchLink><br /><searchLink fieldCode="DE" term="%22Community+Health+Services%22">Community Health Services</searchLink><br /><searchLink fieldCode="DE" term="%22Componential+Analysis%22">Componential Analysis</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10488-023-01320-8
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0894-587X<br />1573-3289
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Multilevel service delivery frameworks are approaches to structuring and organizing a spectrum of evidence-based services and supports, focused on assessment, prevention, and intervention designed for the local context. Exemplar frameworks in child mental health include positive behavioral interventions and supports in education, collaborative care in primary care, and systems of care in community mental health settings. Yet, their high-quality implementation has lagged. This work proposes a conceptual foundation for multilevel service delivery frameworks spanning diverse mental health service settings that can inform development of strategic implementation supports. We draw upon the existing literature for three exemplar multilevel service delivery frameworks in different child mental health service settings to (1) identify core components common to each framework, and (2) to highlight prominent implementation determinants that interface with each core component. Six interrelated components of multilevel service delivery frameworks were identified, including, (1) a systems-level approach, (2) data-driven problem solving and decision-making, (3) multiple levels of service intensity using evidence-based practices, (4) cross-linking service sectors, (5) multiple providers working together, including in teams, and (6) built-in implementation strategies that facilitate delivery of the overall model. Implementation determinants that interface with core components were identified at each contextual level. The conceptual foundation provided in this paper has the potential to facilitate cross-sector knowledge sharing, promote generalization across service settings, and provide direction for researchers, system leaders, and implementation intermediaries/practitioners working to strategically support the high-quality implementation of these frameworks.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1434362
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1434362
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10488-023-01320-8
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 172
    Subjects:
      – SubjectFull: Mental Health
        Type: general
      – SubjectFull: Health Services
        Type: general
      – SubjectFull: Context Effect
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Positive Behavior Supports
        Type: general
      – SubjectFull: Primary Health Care
        Type: general
      – SubjectFull: Community Health Services
        Type: general
      – SubjectFull: Componential Analysis
        Type: general
    Titles:
      – TitleFull: Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks across Child Mental Health Service Settings
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Stephanie A. Moore
      – PersonEntity:
          Name:
            NameFull: Jennifer McGrory Cooper
      – PersonEntity:
          Name:
            NameFull: JoAnne Malloy
      – PersonEntity:
          Name:
            NameFull: Aaron R. Lyon
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0894-587X
            – Type: issn-electronic
              Value: 1573-3289
          Numbering:
            – Type: volume
              Value: 51
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Administration and Policy in Mental Health and Mental Health Services Research
              Type: main
ResultId 1