View in EDS HTML Full Text PDF Full Text

Practice-Driven Research for Statewide Scale Up: Implementation Outcomes of the California Autism Professional Training and Information Network

Saved in:
Bibliographic Details
Title: Practice-Driven Research for Statewide Scale Up: Implementation Outcomes of the California Autism Professional Training and Information Network
Language: English
Authors: Suhrheinrich, Jessica (ORCID 0000-0003-1614-5215), Nahmias, Allison S. (ORCID 0000-0003-4678-0367), Yu, Yue, Melgarejo, Melina (ORCID 0000-0002-2423-7710), Schetter, Patricia, Holt, Tana C., Stahmer, Aubyn C. (ORCID 0000-0002-1596-9848)
Source: Autism: The International Journal of Research and Practice. Apr 2022 26(3):727-736.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Peer Reviewed: Y
Page Count: 10
Publication Date: 2022
Sponsoring Agency: National Institute of Mental Health (NIMH) (DHHS/NIH)
Institute of Education Sciences (ED)
Contract Number: K01MH109574
R324A170063
Document Type: Journal Articles
Reports - Research
Descriptors: Evidence Based Practice, Autism, Pervasive Developmental Disorders, Professional Training, Program Effectiveness, Students with Disabilities, Professional Development, Special Education, Public Schools, Special Education Teachers, Paraprofessional School Personnel, School Personnel, Knowledge Level, Educational Environment
Geographic Terms: California
DOI: 10.1177/13623613211068224
ISSN: 1362-3613
1461-7005
Abstract: Scaling up the use of evidence-based practice (EBP) for autism across service sectors and regions has presented a considerable challenge indicating a clear need for continued development. The California Autism Professional Training and Information Network (CAPTAIN) integrates implementation drivers into specific procedures and methodology as an implementation strategy to support statewide scale up. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, and knowledge, fidelity and use of autism EBPs, and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. CAPTAIN-trained providers reported more openness to EBP. Significantly more CAPTAIN-trained direct service providers reported collecting fidelity of implementation data (X[superscript 2](2, N = 1515) = 10.95, p = 0.004), collecting student data (X[superscript 2](2, N = 1509) = 14.19, p = 0.001), and reported using their primary EBP with "most or all students" (X[superscript 2](2, N = 1514) = 11.41, p = 0.003) than providers not trained by CAPTAIN. In summary, these preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2022
Accession Number: EJ1333832
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwF-9-JWALlIOK48QfEY1WnVAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDOK6YzBAosnZtMTxZQIBEICBmpPci9Th-rrPi-rWxb0igBExM2rJZtoj7XBYmgBAII-WXPKeKYYDJ-YDYPQ0rRLF4c9Fa8FVG2oWkCWydPEL5aj5Xq2q4S03ENgOVzAo_hghi5M-XU-BcWS3MevmwjnTd9LFYZi-VuAPdCG1ExHSzadzFdbJD-BIVaznShYDluEiT4nBi9Ia79MI8Zi2_59duyxq5pcxWATUw_0=
Text:
  Availability: 1
  Value: <anid>AN0155828398;f9d01apr.22;2022Mar22.02:36;v2.2.500</anid> <title id="AN0155828398-1">Practice-driven research for statewide scale up: Implementation outcomes of the California Autism Professional Training and Information Network </title> <p>Scaling up the use of evidence-based practice (EBP) for autism across service sectors and regions has presented a considerable challenge indicating a clear need for continued development. The California Autism Professional Training and Information Network (CAPTAIN) integrates implementation drivers into specific procedures and methodology as an implementation strategy to support statewide scale up. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, and knowledge, fidelity and use of autism EBPs, and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. CAPTAIN-trained providers reported more openness to EBP. Significantly more CAPTAIN-trained direct service providers reported collecting fidelity of implementation data (χ<sup>2</sup>(<reflink idref="bib2" id="ref1">2</reflink>, N = 1515) = 10.95, p = 0.004), collecting student data (χ<sup>2</sup>(<reflink idref="bib2" id="ref2">2</reflink>, N = 1509) = 14.19, p = 0.001), and reported using their primary EBP with "most or all students" (χ<sup>2</sup>(<reflink idref="bib2" id="ref3">2</reflink>, N = 1514) = 11.41, p = 0.003) than providers not trained by CAPTAIN. In summary, these preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level. Supporting use of evidence-based practice in public service programs for autistic individuals is critical. The California Autism Professional Training and Information Network (CAPTAIN) brings together best practices from intervention and implementation research to support scale up of autism services. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, knowledge, fidelity, and use of autism EBPs and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. These preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.</p> <p>Keywords: education services; implementation science; professional development; scale up</p> <p>Access to high-quality school services for students with autism is critical, as over 90% of children with autism are primarily served in public schools ([<reflink idref="bib6" id="ref4">6</reflink>]; [<reflink idref="bib32" id="ref5">32</reflink>]). In California's public schools, autism is now the third largest qualifying disability for special education services, with over 132,359 students (16% of the total population of students with disabilities) receiving services ([<reflink idref="bib7" id="ref6">7</reflink>]). Federal legislation specifies that school practices must be supported by scientifically based evidence and professional wisdom ([<reflink idref="bib12" id="ref7">12</reflink>]; [<reflink idref="bib18" id="ref8">18</reflink>]). Systematic literature reviews identify several evidence-based practices (EBPs) for students with autism ([<reflink idref="bib34" id="ref9">34</reflink>]). Unfortunately, these interventions have not historically been incorporated into classroom practice (e.g. [<reflink idref="bib17" id="ref10">17</reflink>]; [<reflink idref="bib22" id="ref11">22</reflink>]; [<reflink idref="bib33" id="ref12">33</reflink>]; [<reflink idref="bib35" id="ref13">35</reflink>]). In recent studies, 50%–97% of teachers self-reported using at least one EBP ([<reflink idref="bib4" id="ref14">4</reflink>]; [<reflink idref="bib9" id="ref15">9</reflink>]). However, even when teachers are attempting to use EBP for autism, they often have low levels of fidelity, or adherence to the intervention procedures ([<reflink idref="bib39" id="ref16">39</reflink>], [<reflink idref="bib38" id="ref17">38</reflink>]).</p> <p>The field of implementation science has developed with a focus on identifying methods to promote the adoption and integration of EBPs into routine care ([<reflink idref="bib10" id="ref18">10</reflink>]). In contrast to intervention research, which focuses on how specific EBPs improve student outcomes, targets of implementation research include acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration (spread or access within the organization), and sustainability of the practice ([<reflink idref="bib29" id="ref19">29</reflink>]). Individual provider-level factors have been indicated as key indicators in implementation of EBP and are highlighted as inner context factors within multiple implementation science frameworks ([<reflink idref="bib2" id="ref20">2</reflink>]; [<reflink idref="bib3" id="ref21">3</reflink>]). Within autism implementation research, there is some evidence of specific factors linked to teacher EBP use, including teacher knowledge and perceived "social validity" ([<reflink idref="bib21" id="ref22">21</reflink>]) and teacher ratings of training quality and "ease of use" of the practice ([<reflink idref="bib36" id="ref23">36</reflink>]).</p> <p>Scaling up the use of EBP across educators, schools, districts, and regions presents an additional challenge. At a system level, education programs targeting implementation strategies or drivers at both the organizational and the individual provider level report greater success than those who do not have implementation plans ([<reflink idref="bib14" id="ref24">14</reflink>]). For example, intentionally targeting competency drivers such as staff selection, effective training and coaching, and leadership support will support the success of the implementation efforts and EBP fidelity. However, most statewide systems have very limited capacity for monitoring these drivers and scaling up interventions in ways that lead to meaningful improvements in student outcomes ([<reflink idref="bib13" id="ref25">13</reflink>]) indicating a clear need for continued development. States have rarely scaled up EBP successfully indicating additional tools and processes are needed to support effective implementation.</p> <p>We have employed implementation frameworks to describe factors related to the initial development and continued growth of the California Autism Professional Training and Information Network (CAPTAIN; [<reflink idref="bib37" id="ref26">37</reflink>]). Although multiple discrete implementation strategies ([<reflink idref="bib28" id="ref27">28</reflink>]) are incorporated within CAPTAIN activities, we have directly targeted interagency collaboration, use of evidence-based training and coaching practices, leader engagement, and data-driven continuous improvement cycles ([<reflink idref="bib36" id="ref28">36</reflink>]). Here we extend the work to provide preliminary evaluation of CAPTAIN model as a potential implementation strategy to support statewide scale up.</p> <hd id="AN0155828398-2">The CAPTAIN model</hd> <p>In 2008, the California Inter-agency Autism Planning Group (IAPG) was created to align efforts and develop a common training curriculum for autism EBPs. The IAPG participated in a school-based technical assistance project through the National Professional Development Center on Autism Spectrum Disorder (NPDC-ASD; [<reflink idref="bib27" id="ref29">27</reflink>]). In California, fidelity to the target EBP increased by 63% on average (44%–85%) and exceeded 80% for four EBP. In addition, all participating students (<emph>n</emph> = 18) made progress on annual goals based on Goal Attainment Scaling ([<reflink idref="bib31" id="ref30">31</reflink>]) with 44% exceeding expected progress ([<reflink idref="bib37" id="ref31">37</reflink>]). Overall program quality, measured by the Autism Program Environmental Rating Scales (APERS) ([<reflink idref="bib26" id="ref32">26</reflink>]) also increased. Based on these positive outcomes, the IAPG expanded to include additional service sectors, and was renamed the California Autism Professional Training and Information Network (CAPTAIN, www.captain.ca.gov), with a focus on disseminating and implementing EBP for individuals impacted by autism.</p> <p>CAPTAIN is organized into 17 regional teams across the state, that develop regional plans for information dissemination about autism and EBPs, promoting effective EBP implementation and cross-agency regional collaboration and coordination. CAPTAIN currently has over 400 members (called Cadre) representing special education, developmental disabilities and family support service sectors, and university programs. All CAPTAIN Cadre are required to provide training to increase awareness of autism and knowledge of EBP, and CAPTAIN Cadre representing special education services have the additional expectation of providing EBP-specific training and implementation coaching to at least three providers or programs per year.</p> <p>In California, special education services are funded through regional special education local plan areas (SELPAs). SELPAs provide special education compliance monitoring as well as training and technical assistance to the over 1100 local education agencies within their respective catchment areas serving students from ages 3 to 22 years. Each of the 132 SELPAs was offered a designated number of CAPTAIN cadre positions based on the number of students they served who were qualified for special education services for Autism (1 cadre member per every 500 identified students with autism). Selected cadre are individuals within the special education system who possess a strong base of knowledge about autism spectrum disorder and have the capacity to train and coach others from within their SELPA catchment area. During the 2018–2019 academic year, 92% of the SELPAs participated in CAPTAIN, with a total of over 200 school-based Cadre actively participating in the required training and coaching activities.</p> <p>Based on recent data collected for CAPTAIN monitoring and quality improvement purposes, we can estimate the impact of CAPTAIN cadre in terms of reach (see [<reflink idref="bib36" id="ref33">36</reflink>] for a methodological description of the annual survey). The majority of the 223 school-based CAPTAIN Cadre reported they met or exceeded expectations during 2018–2019. Overall, Cadre reported completing over 1500 trainings and over 350 hours of coaching. On average, they trained 88.21 (<emph>SD</emph> = 96.32) providers (including special educators, paraeducators, general educators, and other direct service providers) and coached 30.32 (<emph>SD</emph> = 58.54) providers. Given these self-reported data, we estimate annual totals of 19,495 providers receiving training and 6701 receiving EBP coaching ([<reflink idref="bib5" id="ref34">5</reflink>]).</p> <p>Preliminary data on CAPTAIN impact are encouraging and suggest further exploration of implementation outcomes associated with the CAPTAIN model. Specifically, we are interested in potential impacts of CAPTAIN at the provider level. The current study aims to (<reflink idref="bib1" id="ref35">1</reflink>) evaluate differences in attitudes toward EBP and use of EBP between direct service providers who had, or had not, been trained by CAPTAIN members, and (<reflink idref="bib2" id="ref36">2</reflink>) evaluate differences in EBP knowledge, EBP fidelity, and overall classroom quality between teachers who had, or had not, been trained by CAPTAIN members.</p> <hd id="AN0155828398-3">Community involvement</hd> <p>This study was conducted with the CAPTAIN as a community partner. All participants were community-based service providers within school programs. In addition, co-author P.S. is a certified behavior analyst and community service provider.</p> <hd id="AN0155828398-4">Methods</hd> <p>This work aimed to evaluate differences in implementation outcomes related to EBPs for autism by comparing providers who received training from CAPTAIN members and providers who had not received training from CAPTAIN. Survey data were collected from providers statewide, across two phases, with unique aims.</p> <hd id="AN0155828398-5">Participants</hd> <p>Phase 1 data were drawn from a statewide survey of administrators and providers serving autistic students in public schools (early intervention to post-secondary) throughout California. The subsample of data analyzed for this study included only direct services providers (<emph>n</emph> = 1543). As seen in Table 1, the majority of participants were Special Education teachers (<emph>n</emph> = 838; 54%) and paraprofessionals (<emph>n</emph> = 252; 16%). The majority of the sample held a master's degree (<emph>n</emph> = 874; 57%), followed by a bachelor's degree (<emph>n</emph> = 461; 30%). Most participants reported extensive hands-on experience working with students with autism (<emph>n</emph> = 836; 54%). A portion of the sample had received EBP training from a CAPTAIN member (<emph>n</emph> = 326; CAPTAIN-trained providers), while the majority of participants had not received training from a CAPTAIN member or were unsure if they had received training (<emph>n</emph> = 1217; non-CAPTAIN-trained providers).</p> <p>Graph</p> <p>Table 1. Demographics of survey participants.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th /><th align="left">Statewide survey<italic>n</italic> (%)</th><th align="left">Follow-up survey<italic>n</italic> (%)</th></tr></thead><tbody><tr><td colspan="3">Job title</td></tr><tr><td> Special education teacher (serving on single school site)</td><td>823 (53.3)</td><td>220 (98.2)</td></tr><tr><td> Paraprofessional</td><td>252 (16.3)</td><td>0 (0)</td></tr><tr><td> SLP/SLPA</td><td>152 (9.9)</td><td>0 (0)</td></tr><tr><td> Psychologist</td><td>150 (9.7)</td><td>0 (0)</td></tr><tr><td> Itinerant special education teacher (serving on multiple school sites)</td><td>34 (2.2)</td><td>2 (0.9)</td></tr><tr><td> General education teacher</td><td>33 (2.1)</td><td>2 (0.9)</td></tr><tr><td> OT/OTA</td><td>32 (2.1)</td><td>0 (0)</td></tr><tr><td> Mental health counselor/social worker</td><td>18 (1.2)</td><td>0 (0)</td></tr><tr><td> Special education teacher (serving on multiple school sites)</td><td>15 (1.0)</td><td>0 (0)</td></tr><tr><td> Specialist (e.g. behavior specialist, autism specialist)</td><td>4 (0.3)</td><td>0 (0)</td></tr><tr><td> Physical therapist</td><td>3 (0.2)</td><td>0 (0)</td></tr><tr><td> Case manager</td><td>1 (0.1)</td><td>0 (0)</td></tr><tr><td> Teacher on special assignment (TOSA)</td><td>2 (0.1)</td><td>0 (0)</td></tr><tr><td> Not reported</td><td>24 (1.6)</td><td>0 (0)</td></tr><tr><td colspan="3">Education</td></tr><tr><td> High school</td><td>32 (2.1)</td><td>0 (0)</td></tr><tr><td> Associate degree</td><td>118 (7.6)</td><td>0 (0)</td></tr><tr><td> Bachelor's degree</td><td>461 (29.9)</td><td>82 (36.6)</td></tr><tr><td> Master's degree</td><td>874 (56.6)</td><td>142 (63.4)</td></tr><tr><td> Doctorate</td><td>34 (2.2)</td><td>0 (0)</td></tr><tr><td> Not reported</td><td>24 (1.6)</td><td>0 (0)</td></tr><tr><td colspan="3">ASD experience</td></tr><tr><td> Extensive</td><td>836 (54.2)</td><td>145 (64.7)</td></tr><tr><td> Moderate</td><td>428 (27.7)</td><td>55 (24.6)</td></tr><tr><td> Some recent</td><td>221 (14.3)</td><td>204 (8.9)</td></tr><tr><td> Some distant</td><td>20 (1.3)</td><td>4 (1.8)</td></tr><tr><td> Little to none</td><td>14 (.9)</td><td>0 (0)</td></tr><tr><td> Not reported</td><td>24 (1.6)</td><td>0 (0)</td></tr></tbody></table> </ephtml> </p> <p>1 ASD: autism spectrum disorder ; SLP: Speech language pathologist; SLPA: Speech language pathology assistant; OT: Occupational therapist; OTA: Occupational therapy assistant..</p> <p>Phase 2 participants included only teacher participants. Teachers who agreed to be contacted for further research received a follow-up survey that asked specifically about classroom characteristics/quality, EBP fidelity, and EBP knowledge. The Phase 2 sample included 224 teachers, including some who had received training by a CAPTAIN member (<emph>n</emph> = 55; CAPTAIN-trained teachers) and some who had not or were unsure (<emph>n</emph> = 169; non-CAPTAIN-trained teachers). The majority were female (<emph>n</emph> = 179, 80%), White (<emph>n</emph> = 162, 72%), special education teachers (<emph>n</emph> = 220; 98%), held a master's degree (<emph>n</emph> = 142; 63%), and reported extensive hands-on experience working with students with autism (<emph>n</emph> = 145; 65%) (see Table 1).</p> <hd id="AN0155828398-6">Procedures</hd> <p>This study was approved by the University of California Davis, Institutional Review Board. During Phase 1, a survey was administered via Qualtrics and distributed through CAPTAIN social media (Facebook and Instagram), email and recruitment postcards (<emph>n</emph> = 4500). CAPTAIN Cadre were asked to distribute the email invitation and recruitment postcards to their school sites, professional organizations, and to teachers and other service providers they worked with directly. Average survey completion was 30 to 45 min. Participants were entered in a 1 in 20 opportunity drawing to win a $50 gift card upon completion of the statewide survey. Recruitment began in May 2018 and ended in March 2020.</p> <p>A Phase 2 follow-up online survey was sent out to participants who selected "teacher" as their job title (e.g. special education teacher, general education teacher, itinerant teacher) on the statewide survey and indicated they were willing to be contacted for future research. This survey asked teachers to report on their classroom practices/quality, knowledge of EBP, and fidelity of EBP implementation for the EBPs they primarily used in their classroom. Average survey completion was 60 min. All participants who completed the follow-up survey received a $50 gift card. Recruitment began in November 2019 and ended in March 2020.</p> <hd id="AN0155828398-7">Measures</hd> <p></p> <hd id="AN0155828398-8">Phase 1—statewide survey measures</hd> <p></p> <hd id="AN0155828398-9">EBP attitude</hd> <p>Respondents completed the Evidence-Based Practice Attitude Scale (EBPAS) ([<reflink idref="bib1" id="ref37">1</reflink>]), a 15-item measure that assesses four general attitudes toward adoption of EBPs: appeal, requirements, openness, and divergence. The EBPAS assesses provider attitudes toward adoption of EBP in public sector service settings and has been used in mental health, medical, school, and social service settings. Items are scored on a 5-point Likert-type scale (0–4) with answers ranging from "Not at all" to "Very great extent." Domain scores were calculated by averaging the item scores in each domain. The EBPAS demonstrates good internal consistency reliability (α = 0.76) and concurrent and predictive validity.</p> <hd id="AN0155828398-10">Report of EBP use</hd> <p>Participants were asked to select all the EBPs they used (out of 27 EBPs; [<reflink idref="bib41" id="ref38">41</reflink>]) in the past week as well as the <emph>primary</emph> EBP they used in the past week with an autistic student. They reported (<reflink idref="bib1" id="ref39">1</reflink>) number of days the EBP was used in the past week (0–5 days); (<reflink idref="bib2" id="ref40">2</reflink>) whether they collected fidelity data on their EBP use (Yes, I meant to but didn't have time, No); (<reflink idref="bib3" id="ref41">3</reflink>) whether they collected student outcome data (Yes, I meant to but didn't have time, No); and (<reflink idref="bib4" id="ref42">4</reflink>) the number of students with autism with whom they used the EBP (One student, Some students, Most or all students). The report of use measure was developed by members of the research team.</p> <hd id="AN0155828398-11">Implementation outcomes</hd> <p>Participants completed an adapted version of the Evidence-Based Practice Outcomes Scale ([<reflink idref="bib11" id="ref43">11</reflink>]) which asked them the extent to which they (<reflink idref="bib1" id="ref44">1</reflink>) use all components of their primary EBP, (<reflink idref="bib2" id="ref45">2</reflink>) have adapted their primary EBP, (<reflink idref="bib3" id="ref46">3</reflink>) feel competent implementing their primary EBP, and (<reflink idref="bib4" id="ref47">4</reflink>) feel knowledgeable explaining their primary EBP. Participants self-rated on a 5-point Likert-type scale (0–4) with answers ranging from "Not at all" to "Very great extent."</p> <hd id="AN0155828398-12">Phase 2—follow-up survey measures</hd> <p></p> <hd id="AN0155828398-13">Fidelity of EBP implementation</hd> <p>Participants reported their fidelity to the components of their primary EBP by completing an implementation checklist. Implementation checklists were pulled from the Autism Focused Intervention Resources & Modules (AFIRM) on the NPDC website (National Professional Development Center on Autism Spectrum Disorder, n.d.). The total number of items on the implementation checklists varied by EBP, but all were divided in to three stages of implementation: Planning, Using, and Monitoring. Participants were asked to check off whether they completed each component on the checklist for their primary EBP. Fidelity was calculated as a percentage of the total number of components completed.</p> <hd id="AN0155828398-14">EBP knowledge</hd> <p>Participants answered true or false and multiple-choice questions assessing their knowledge of their primary EBP. Knowledge surveys were pulled from the Autism Focused Intervention Resources & Modules (AFIRM) on the NPDC website (https://afirm.fpg.unc.edu/afirm-modules, [<reflink idref="bib23" id="ref48">23</reflink>]). The total number of knowledge survey items varied across specific EBP. The percent of correct responses was used for analysis.</p> <hd id="AN0155828398-15">Classroom quality</hd> <p>Participants completed a self-report version of the Autism Program Environmental Rating Scale (APERS) designed to assess the overall quality of program environments for students with autism. The APERS Self-Assessment Tool ([<reflink idref="bib23" id="ref49">23</reflink>]) consists of 64 items across 11 domains for the preschool/elementary form and 66 items across 11 domains for the middle school/high school form. Participants completed the version that aligned with the primary age group they taught. Domains include Learning Environments, Classroom Structure and Schedule, Positive Learning Climate, Assessment, Curriculum and instruction, Communication, Social Competence, Personal Independence and Competence, Functional Behavior, Family Involvement, and Teaming. Items are scored on a 3-point Likert-type scale with answers ranging from "1 = This is a challenge for our program" to "3 = This element is consistently in place, but we still have some work to do" to "5 = This is a real strength for our program." Classroom quality was calculated based on the average score for each domain.</p> <hd id="AN0155828398-16">Data analysis</hd> <p>Data were analyzed using IBM SPSS Statistics. Statistical analysis varied depending upon the data form. Data were examined for normality and appropriate transformations were applied for highly skewed and kurtotic data based on the recommendations of [<reflink idref="bib40" id="ref50">40</reflink>]. Means, standard deviations, and frequencies were calculated to describe the data. Independent <emph>t</emph> tests and chi-square analyses were used to examine group differences between CAPTAIN-trained providers and non-CAPTAIN-trained providers in report of use. Independent <emph>t</emph> tests were used to examine group differences between CAPTAIN-trained providers and non-CAPTAIN-trained providers in EBP attitude, implementation outcomes, classroom outcomes, fidelity of EBP implementation, and knowledge of EBP.</p> <hd id="AN0155828398-17">Results</hd> <p>Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers.</p> <hd id="AN0155828398-18">Phase 1—statewide survey results</hd> <p></p> <hd id="AN0155828398-19">EBP attitude</hd> <p>Overall, participants reported moderate scores on the Openness (<emph>M</emph> = 3.13, <emph>SD</emph> = 0.76), Appeal (<emph>M</emph> = 3.32, <emph>SD</emph> = 0.72), and Requirements (<emph>M</emph> = 2.99, <emph>SD</emph> = 0.96) subscales of the EBPAS, and relatively lower scores on the Divergence subscale (<emph>M</emph> = 2.30, <emph>SD</emph> = 0.67). There was a significant group difference in EBP Openness between CAPTAIN-trained providers versus non-CAPTAIN-trained providers, <emph>t</emph>(<reflink idref="bib579" id="ref51">579</reflink>) = −3.29, <emph>p</emph> = 0.001. Specifically, CAPTAIN-trained providers reported significantly higher levels of EBP Openness (<emph>M</emph> = 3.24, <emph>SD</emph> = 0.68) than non-CAPTAIN-trained providers (<emph>M</emph> = 3.09, <emph>SD</emph> = 0.77). No significant associations were found for the Appeal, Divergence, and Requirement subscales (<emph>p</emph> > 0.05) (see Table 2).</p> <p>Graph</p> <p>Table 2. Outcomes in CAPTAIN-trained and Non-CAPTAIN-trained providers.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left" rowspan="2">Provider outcomes</th><th align="left" colspan="2">CAPTAIN trained</th><th align="left" rowspan="2">Statistics</th><th align="left" rowspan="2"><italic>p</italic></th></tr><tr><th align="left">Yes</th><th align="left">No</th></tr></thead><tbody><tr><td colspan="5">Direct service providers and teachers</td></tr><tr><td colspan="5"> Report of use</td></tr><tr><td> Use primary EBP with most or all students</td><td>69.4%</td><td>62.0%</td><td>χ2(2) = 11.4</td><td>0.003</td></tr><tr><td> Days/week use of primary EBP</td><td>Mean 3.6</td><td>Mean 3.5</td><td><italic>t</italic>(570) = 1.8</td><td>0.078</td></tr><tr><td> Collect fidelity data on primary EBP</td><td>42.6%</td><td>35.6%</td><td>χ2(2) = 10.9</td><td>0.004</td></tr><tr><td> Collect student data on EBP response</td><td>55.2%</td><td>44.3%</td><td>χ2(2) = 14.2</td><td>0.001</td></tr><tr><td colspan="5"> Implementation Outcomes Scale</td></tr><tr><td> Use all components of primary EBP (max 4)</td><td>Mean 2.8</td><td>Mean 2.6</td><td><italic>t</italic>(555) =–2.9</td><td>0.004</td></tr><tr><td> Competence implementing primary EBP (max 4)</td><td>Mean 2.9</td><td>Mean 2.7</td><td><italic>t</italic>(527) =–3.7</td><td><0.001</td></tr><tr><td> Knowledge explaining primary EBP (max 4)</td><td>Mean 2.6</td><td>Mean 2.4</td><td><italic>t</italic>(533) =–3.6</td><td><0.001</td></tr><tr><td> Adapted primary EBP (max 4)</td><td>Mean 2.4</td><td>Mean 2.3</td><td><italic>t</italic>(1443) = 1.6</td><td>0.111</td></tr><tr><td colspan="5"> EBPAS</td></tr><tr><td> Openness (max 4)</td><td>Mean 3.2</td><td>Mean 3.1</td><td><italic>t</italic>(579) =–3.3</td><td>0.001</td></tr><tr><td> Appeal (max 4)</td><td>Mean 3.4</td><td>Mean 3.3</td><td><italic>t</italic>(563) =–1.6</td><td>0.118</td></tr><tr><td> Divergence (max 4)</td><td>Mean 2.3</td><td>Mean 2.3</td><td><italic>t</italic>(548) =–1.6</td><td>0.114</td></tr><tr><td> Required (max 4)</td><td>Mean 3.0</td><td>Mean 3.0</td><td><italic>t</italic>(1282) =–0.7</td><td>0.486</td></tr><tr><td colspan="5">Teachers</td></tr><tr><td> NPDC EBP Knowledge Assessments % correct for primary EBP</td><td>67.2%</td><td>56.9%</td><td><italic>t</italic>(222) =–2.5</td><td>0.012</td></tr><tr><td> Self-report fidelity of implementation checklist % fidelity for primary EBP</td><td>88.9%</td><td>86.3%</td><td><italic>t</italic>(219) = 0.5</td><td>0.583</td></tr><tr><td> APERS Learning Environment (max 5)</td><td>Mean 3.9</td><td>Mean 3.7</td><td><italic>t</italic>(202) =–2.1</td><td>0.041</td></tr></tbody></table> </ephtml> </p> <p>2 CAPTAIN: California Autism Professional Training and Information Network; EBP: evidence-based practice; EBPAS: Evidence-Based Practice Attitude Scale; NPDC: National Professional Development Center on Autism Spectrum Disorder; APERS: Autism Program Environmental Rating Scale.</p> <hd id="AN0155828398-20">Report of EBP use</hd> <p>Regarding the total number of EBPs used, CAPTAIN-trained providers reported on average using 9.29 EBPs (<emph>SD</emph> = 5.02) and non-CAPTAIN-trained providers reported 9.02 EBPs (<emph>SD</emph> = 5.05), with no significant difference between the two groups, <emph>t</emph>(1541) = 0.87, <emph>p</emph> = 0.38. The most frequently used primary EBP reported by CAPTAIN-trained providers was Visual Supports (22%), followed by Reinforcement (20%), and Prompting (10%); whereas the most frequently used primary EBP reported by providers not trained by CAPTAIN was Reinforcement (20%), followed by Prompting (16%), and Visual Supports (14%).</p> <p>The majority of the providers reported using their primary EBP 4–5 days in the past week (75%) while about 12% reported 3 days, 7% reported 2 days, about 5% reported 1 day, and 1% reported 0 days in the last week. About a third (36%) of the providers reported collecting fidelity data. Close to half of the providers (46%) reported collecting student data, and 62% reported using their primary EBP with most or all students.</p> <p>When investigating the impact of being trained by CAPTAIN, results indicated that a larger proportion of CAPTAIN-trained providers reported collecting fidelity data (χ<sups>2</sups>(<reflink idref="bib2" id="ref52">2</reflink>, _I_N_i_ = 1191) = 10.95, <emph>p</emph> = 0.004), collecting student data (χ<sups>2</sups>(<reflink idref="bib2" id="ref53">2</reflink>, _I_N_i_ = 1185) = 14.19, <emph>p</emph> = 0.001), and reported using their primary EBP with "most or all students" (χ<sups>2</sups>(<reflink idref="bib2" id="ref54">2</reflink>, _I_N_i_ = 1514) = 11.41, <emph>p</emph> = 0.003) than providers not trained by CAPTAIN. Frequency of primary EBP use did not differ significantly between CAPTAIN and non-CAPTAIN providers (<emph>t</emph> (<reflink idref="bib570" id="ref55">570</reflink>) = −1.76, <emph>p</emph> = 0.08) (see Table 2).</p> <hd id="AN0155828398-21">Implementation outcomes</hd> <p>The implementation outcomes associated with providers' primary EBP were moderate across all items: use of all components of EBP (<emph>M</emph> = 2.63, <emph>SD</emph> = 0.97), adaptation of EBP (<emph>M</emph> = 2.34, <emph>SD</emph> = 1.06), feeling competent in implementing EBP (<emph>M</emph> = 2.70, <emph>SD</emph> = 0.97), and feeling knowledgeable explaining EBP (<emph>M</emph> = 2.45, <emph>SD</emph> = 1.07).</p> <p>There were significant group differences between CAPTAIN-trained providers and non-CAPTAIN-trained providers for several implementation outcomes. Specifically, CAPTAIN-trained providers reported higher levels of using all components of the primary EBP (<emph>M</emph> = 2.76, <emph>SD</emph> = 0.88) than non-CAPTAIN-trained providers (<emph>M</emph> = 2.59, <emph>SD</emph> = 0.98, <emph>t</emph> (<reflink idref="bib555" id="ref56">555</reflink>) = −2.91, <emph>p</emph> = 0.004); CAPTAIN-trained providers reported higher levels of feeling competent implementing EBPs (<emph>M</emph> = 2.88, <emph>SD</emph> = 0.92) than non-CAPTAIN-trained providers (<emph>M</emph> = 2.65, <emph>SD</emph> = 0.98, <emph>t</emph> (<reflink idref="bib527" id="ref57">527</reflink>) = −3.75, <emph>p</emph> < 0.001); CAPTAIN-trained providers reported higher levels of feeling knowledgeable explaining their primary EBP (<emph>M</emph> = 2.63, <emph>SD</emph> = 1.01) than providers not trained by CAPTAIN (<emph>M</emph> = 2.40, <emph>SD</emph> = 1.08, <emph>t</emph> (<reflink idref="bib533" id="ref58">533</reflink>) = −3.60, <emph>p</emph> < 0.001) (see Table 2).</p> <p>No significant differences were found between CAPTAIN-trained providers and non-CAPTAIN-trained providers on the adaptation of EBP item (<emph>p</emph> > 0.05).</p> <hd id="AN0155828398-22">Follow-up survey results</hd> <p></p> <hd id="AN0155828398-23">Fidelity of EBP implementation</hd> <p>The mean percentage of specific EBP fidelity for teachers' primary EBP was 87% (<emph>N</emph> = 221; <emph>SD</emph> = 15.31%). There was no significant group difference in the self-reported fidelity of teachers' primary EBP between CAPTAIN-trained and non-CAPTAIN-trained teachers, <emph>t</emph> (<reflink idref="bib219" id="ref59">219</reflink>) = 0.55, <emph>p</emph> = 0.583 (see Table 2).</p> <hd id="AN0155828398-24">EBP knowledge</hd> <p>The mean percentage of correct responses on specific EBP knowledge for teachers' primary EBP was 59.42% (<emph>N</emph> = 224; <emph>SD</emph> = 26.26%). CAPTAIN-trained teachers had significantly higher EBP knowledge (<emph>M</emph> = 67.15%, <emph>SD</emph> = 25.53%) than teachers not trained by CAPTAIN (<emph>M</emph> = 56.90%, <emph>SD</emph> = 26.08%), <emph>t</emph>(<reflink idref="bib222" id="ref60">222</reflink>) = −2.54, <emph>p</emph> = 0.012.</p> <hd id="AN0155828398-25">Classroom quality</hd> <p>There was a significant group difference between CAPTAIN-trained teachers versus non-CAPTAIN-trained teachers regarding the learning environment subscale. Specifically, CAPTAIN-trained teachers reported higher levels in learning environment (<emph>mean<subs>CAPTAIN</subs></emph> = 3.92, <emph>SD<subs>CAPTAIN</subs></emph> = 0.77; <emph>mean<subs>NonCAPTAIN</subs></emph> = 3.65, <emph>SD<subs>NonCAPTAIN</subs></emph> = 0.82), <emph>t</emph> (<reflink idref="bib202" id="ref61">202</reflink>) = −2.05, <emph>p</emph> = 0.041. No significant associations were found for all other APERS subscales or the overall score (<emph>p</emph> > 0.05).</p> <hd id="AN0155828398-26">Discussion</hd> <p>The use of research-based practices is mandated by IDEA and ESSA and has been linked to best outcomes for students with autism, which highlights effective implementation and scale up of EBPs in schools as a critical priority. The growing literature on factors that support the implementation process indicates key drivers which can be considered targets of implementation interventions to improve implementation outcomes. In this study, we explored implementation outcomes at the direct service provider level and evaluated differences between CAPTAIN-trained and non-CAPTAIN-trained providers using one of the first large-scale statewide examinations across multiple levels of the special education service system. Overall outcomes indicate CAPTAIN-trained providers and teachers report more favorable attitudes toward EBP, better implementation outcomes related to data collection, and use with students, higher knowledge of their primary EBP, and better ratings of learning environment. These findings show great promise for CAPTAIN as a model to support statewide scale up if EBP for autism and are discussed in more detail below.</p> <p>One factor that has been linked to positive implementation outcomes is individual provider attitudes toward EBPs ([<reflink idref="bib1" id="ref62">1</reflink>]; [<reflink idref="bib2" id="ref63">2</reflink>]; [<reflink idref="bib30" id="ref64">30</reflink>]). This intuitively makes sense, as it is important that providers are open to learning about and using these EBPs prior to adoption and effective implementation. Multiple studies have found that provider attitudes before training, especially openness to the use of evidence-based practice and perceptions of the appeal of the practice, are linked to fidelity to the intervention after training ([<reflink idref="bib2" id="ref65">2</reflink>]; [<reflink idref="bib3" id="ref66">3</reflink>]). Furthermore, attitudes toward a specific practice have been linked to reported use of that practice ([<reflink idref="bib30" id="ref67">30</reflink>]) and negative beliefs about a practice may be a barrier to adoption (e.g. [<reflink idref="bib16" id="ref68">16</reflink>]) In addition, it has been suggested that measures of provider attitude, such as the EBPAS, could be applied in the education sector to examine the impact of implementation interventions ([<reflink idref="bib8" id="ref69">8</reflink>]). Our findings reveal that CAPTAIN-trained providers were much more open to EBP use than those who were not CAPTAIN trained. This may be a result of their interaction with a CAPTAIN trainer, or it may be that those who were more open initially sought out the type of training being offered by CAPTAIN. Openness to EBPs may lead to these teachers seeking out additional EBP trainings in their future professional development, thus further expanding their use of effective practices. Determining if openness can be impacted by interactions like those with CAPTAIN trainers is an area for further study.</p> <p>One of the primary goals of CAPTAIN is to increase provider knowledge of EBPs as an initial step toward implementation. It is encouraging that CAPTAIN-trained teachers not only scored significantly higher on EBP knowledge assessments, but also felt confident explaining the EBPs to others. Passing on EBP knowledge to other team members who may have limited access to professional development, such as paraeducators, is essential and can help to promote the spread and scale up of EBP. EBP use can be conceptualized as adherence to protocol (fidelity) as well as dosage (frequency of use) and reach (number of students receiving the intervention). Using EBPs with high fidelity, with moderate to high dosage across many students will likely maximize student impact, so these are important measures to consider. CAPTAIN-trained providers reported using fidelity checklists to monitor their own implementation at higher rates than the other providers and also reported greater use of all components of their primary EBPs. Self-monitoring of implementation fidelity could be an effective way to prevent implementation drift that often occurs following an initial period of high integrity to an EBP. Because using EBP checklists to monitor fidelity is an essential component of the CAPTAIN model of coaching, perhaps this influenced providers to self-monitor their fidelity after coaching ended. There is some research to suggest that self-monitoring of fidelity provides a practical and effective approach to maintaining EBP fidelity over time (e.g. [<reflink idref="bib24" id="ref70">24</reflink>]). However, additional research is needed to examine the validity and accuracy of self-recorded EBP fidelity as well as the role of this process in sustainment. A greater proportion of CAPTAIN-trained providers reported using their primary EBP with most or all students, thus suggesting that they may generalize use of EBPs classroom wide, across students. In addition, their report of the learning environment classroom quality indicator from the APERS was also significantly higher.</p> <p>This study suggests that the CAPTAIN model can support scale of EBP in special education and meet the significant needs of teachers supporting autistic students. Next steps to improving scale up efforts to meet state needs will involve better understanding the mechanisms by which CAPTAIN improves EBP fidelity and use for educators. For example, data indicate that participating with CAPTAIN trainers may provide important social support networks for educators attempting to implement EBP ([<reflink idref="bib20" id="ref71">20</reflink>]). In addition, it may be helpful to better understand how training in EBP fidelity tools may support EBP fidelity. While all educators had access to the AFIRM fidelity tools, those trained by CAPTAIN received explicit instruction on how to use the tools, which may have increased their understanding and expectations for use</p> <p>These promising findings support the potential use of the CAPTAIN model for successful EBP scale up; however, there are several limitations to the current study. One limitation of the study is related to characterizing the reach of recruitment efforts and representativeness of the sample participants. Our primary recruitment strategies involved social media distribution and broad distribution through email with requests that the recruitment information be forwarded to others within the educational sector. Therefore, accurate measurement of the rate of response to recruitment request and the representativeness of the sample are not feasible. Related to the sample, our current data set did not allow for analysis of specific outcomes at the individual practice level. That is, there was no sufficient power to compare results between individual EBP. Another primary limitation of this study is that data were collected through provider self-report. This may contribute to the lack of significant difference on specific EBP fidelity measures, in that self-report scores were overall very high. Future studies should assess fidelity using more objective measures, and could potentially examine whether self-monitoring of EBP is accurate and helps to sustain EBP use over time. Additional objective measures, such as independent program evaluation, would be helpful in further evaluating the impact of CAPTAIN on EBP use and overall classroom quality. Finally, the associations presented should be interpreted cautiously and with the understanding that there are no available provider measures prior to CAPTAIN training.</p> <p>In summary, these preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level. Future research will involve objective assessment of teacher and student outcomes that result from CAPTAIN participation.</p> <ref id="AN0155828398-27"> <title> References </title> <blist> <bibl id="bib1" idref="ref35" type="bt">1</bibl> <bibtext> Aarons G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61–74. https://doi.org/10.1023/b:mhsr.0000024351.12294.65</bibtext> </blist> <blist> <bibl id="bib2" idref="ref1" type="bt">2</bibl> <bibtext> Aarons G. A., Hurlburt M., Horwitz S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7</bibtext> </blist> <blist> <bibl id="bib3" idref="ref21" type="bt">3</bibl> <bibtext> Beidas R. S., Edmunds J., Ditty M., Watkins J., Walsh L., Marcus S., Kendall P. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 788–799. https://doi.org/10.1007/s10488-013-0529-x</bibtext> </blist> <blist> <bibl id="bib4" idref="ref14" type="bt">4</bibl> <bibtext> Brock M. E., Dynia J. M., Dueker S. A., Barczak M. A. (2020). Teacher-reported priorities and practices for students with autism: Characterizing the research-to-practice gap. Focus on Autism & Other Developmental Disabilities, 35, 67–78. https://doi.org/10.1177/1088357619881217</bibtext> </blist> <blist> <bibl id="bib5" idref="ref34" type="bt">5</bibl> <bibtext> Brookman-Frazee L., Baker-Ericzén M., Chan J., Dickson K., Rieth S., Haine-Schlagel R., Stadnick N., Stahmer A., Suhrheinrich J. (in press). Applying dissemination and implementation science to facilitate community implementation of evidence-based interventions. Handbook of autism and pervasive developmental disorder. Springer.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext> Brookman-Frazee L., Baker-Ericzén M., Stahmer A., Mandell D., Haine R. A., Hough R. L. (2009). Involvement of youths with autism spectrum disorders or intellectual disabilities in multiple public service systems. Journal of Mental Health Research in Intellectual Disabilities, 2(3), 201–219. https://doi.org/10.1080/19315860902741542</bibtext> </blist> <blist> <bibl id="bib7" idref="ref6" type="bt">7</bibl> <bibtext> California Department of Education. (n.d.). California's longitudinal pupil achievement data system. https://<ulink href="http://www.calpads.org">www.calpads.org</ulink></bibtext> </blist> <blist> <bibl id="bib8" idref="ref69" type="bt">8</bibl> <bibtext> Cook C. R., Davis C., Brown E. C., Locke J., Ehrhart M. G., Aarons G. A., Larson M., Lyon A. R. (2018). Confirmatory factor analysis of the Evidence-Based Practice Attitudes Scale with school-based behavioral health consultants. Implementation Science, 13(1), Article 116. https://doi.org/10.1186/s13012-018-0804-z</bibtext> </blist> <blist> <bibl id="bib9" idref="ref15" type="bt">9</bibl> <bibtext> Dynia J. M., Walton K., Brock M. E., Tiede G. (2020). Early childhood special education teachers' use of evidence-based practices with children with autism spectrum disorder. Research in Autism Spectrum Disorders, 77, Article 101606. https://doi.org/10.1016/j.rasd.2020.101606</bibtext> </blist> <blist> <bibtext> Eccles M.P., Mittman B.S. (2006) Welcome to Implementation Science. Implementation Sci1, 1. https://doi.org/10.1186/1748-5908-1-1</bibtext> </blist> <blist> <bibtext> Ehrhart M. G., Aarons G. A., Farahnak L. R. (2015). Going above and beyond for implementation: The development and validity testing of the Implementation Citizenship Behavior Scale (ICBS). Implementation Science: IS, 10, Article 65. https://doi.org/10.1186/s13012-015-0255-8</bibtext> </blist> <blist> <bibtext> Every Student Succeeds Act. (2015). Public Law 114—95—GovInfo. https://<ulink href="http://www.govinfo.gov/app/details/PLAW-114publ95">www.govinfo.gov/app/details/PLAW-114publ95</ulink></bibtext> </blist> <blist> <bibtext> Fixsen D. L., Blasé K. A., Metz A., Dyke M. V. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230. https://doi.org/10.1177/001440291307900206</bibtext> </blist> <blist> <bibtext> Fixsen D. L., Blasé K. A., Timbers G. D., Wolf M. M. (2007). In search of program implementation: 792 replications of the Teaching-Family Model. The Behavior Analyst Today, 8(1), 96–110. https://doi.org/10.1037/h0100104</bibtext> </blist> <blist> <bibtext> Green A. E., Albanese B. J., Shapiro N. M., Aarons G. A. (2014). The roles of individual and organizational factors in burnout among community-based mental health service providers. Psychological Services, 11(1), 41–49. https://doi.org/10.1037/a0035299</bibtext> </blist> <blist> <bibtext> Harned M. S., Dimeff L. A., Woodcock E. A., Contreras I. (2013). Predicting adoption of exposure therapy in a randomized controlled dissemination trial. Journal of Anxiety Disorders, 27(8), 754–762.</bibtext> </blist> <blist> <bibtext> Hess K., Morrier M., Heflin L., Ivey M. (2008). Autism treatment survey: Services received by children with autism spectrum disorders in public school classrooms. Journal of Autism and Developmental Disorders, 38, 961–971. https://doi.org/10.1007/s10803-007-0470-5</bibtext> </blist> <blist> <bibtext> Individuals With Disabilities Education Act. (2004). Public Law 101-476, GovInfo. https://<ulink href="http://www.govinfo.gov/app/details/CREC-2004-11-20/CREC-2004-11-20-pt1-PgS11669">www.govinfo.gov/app/details/CREC-2004-11-20/CREC-2004-11-20-pt1-PgS11669</ulink></bibtext> </blist> <blist> <bibtext> Jobin A., Stahmer A., Dickson K. S., Nahmias A., Chlebowski C., Brookman-Frazee L. (2018, November). Characterization of inner context implementation factors related to the implementation of evidence-based interventions for children with ASD: Comparing implementation climate and leadership in education and mental health settings [Conference session]. In K. S. Dickson (Chair), Examination of Implementation Leadership and Climate on Implementation in Schools and Community Mental Health Services. Symposium conducted at the 52nd Annual Convention of the Association for Behavioral and Cognitive Therapies, Washington, DC, United States.</bibtext> </blist> <blist> <bibtext> McGhee Hassrick E., Freidman C., Schetter P., Melgarejo M., Nahmias A., Li J., Suhrheinrich J., Stahmer A. (2020, January). The impact of social networks on scaling up evidence-based autism practices in schools [Poster presentation]. Annual Principal Investigators Meeting for the Institute for Education Sciences, Washington, DC, United States.</bibtext> </blist> <blist> <bibtext> McNeill J. (2019). Social validity and teachers' use of evidence-based practices for autism. Journal of Autism and Developmental Disorders, 49, 4585–4594. https://doi.org/10.1007/s10803-019-04190-y</bibtext> </blist> <blist> <bibtext> Morrier M. J., Hess K. L., Heflin L. J. (2010). Teacher training for implementation of teaching strategies for students with autism spectrum disorders. Teacher Education and Special Education, 34, 119–132. https://doi.org/10.1177/0888406410376660</bibtext> </blist> <blist> <bibtext> National Professional Development Center on Autism Spectrum Disorder. (2011). https://autismpdc.fpg.unc.edu/national-professional-development-center-autism-spectrum-disorder</bibtext> </blist> <blist> <bibtext> Nelson J. R., Oliver R. M., Hebert M. A., Bohaty J. (2015). Use of self-monitoring to maintain program fidelity of multi-tiered interventions. Remedial and Special Education, 36(1), 14–19. https://doi.org/10.1177/0741932514544970</bibtext> </blist> <blist> <bibtext> Novins D. K., Green A. E., Legha R. K., Aarons G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52(10), 1009–1025.e1018. https://doi.org/10.1016/j.jaac.2013.07.012</bibtext> </blist> <blist> <bibtext> Odom S. L., Cox A., Sideris J. H., Hume K. A., Hedges S., Kucharczyk S., Shaw E., Boyd B. A., Reszka S. S., Neitzel J. (2018). Assessing quality of program environments for children and youth with autism: Autism Program Environment Rating Scale (APERS). Journal of Autism and Developmental Disorders, 48, 913–924. https://doi.org/10.1007/s10803-017-3379-7</bibtext> </blist> <blist> <bibtext> Odom S. L., Cox A. W., Brock M. E. (2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79(2), 233–251. <ulink href="http://libproxy.sdsu.edu/login?url=https://www-proquest-com.libproxy.sdsu.edu/scholarly-journals/implementation-science-professional-development/docview/1270781690/se-2?accountid=13758">http://libproxy.sdsu.edu/login?url=https://www-proquest-com.libproxy.sdsu.edu/scholarly-journals/implementation-science-professional-development/docview/1270781690/se-2?accountid=13758</ulink></bibtext> </blist> <blist> <bibtext> Powell B. J., Waltz T. J., Chinman M. J., Damschroder L. J., Smith J. L., Matthieu M. M., Kirchner J. E. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 1–14.</bibtext> </blist> <blist> <bibtext> Proctor E., Silmere H., Raghavan R., Hovmand P., Aarons G., Bunger A., Hensley M. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.</bibtext> </blist> <blist> <bibtext> Reding M. E. J., Chorpita B. F., Lau A. S., Innes-Gomberg D. (2013). Providers' attitudes toward evidence-based practices: Is it just about providers, or do practices matter, too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1</bibtext> </blist> <blist> <bibtext> Ruble L., McGrew J. H., Toland M. D. (2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42(9), 1974–1983. https://doi.org/10.1007/s10803-012-1446-7</bibtext> </blist> <blist> <bibtext> Snyder T. D., De Brey C., Dillow S. A. (2019). Digest of education statistics 2018 (NCES 2020-009) (54th ed.). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. https://nces.ed.gov/pubs2020/2020009.pdf</bibtext> </blist> <blist> <bibtext> Stahmer A. C., Ingersoll B. (2004). Inclusive programming for toddlers with autism spectrum disorders. Journal of Positive Behavior Interventions, 6(2), 67–82. https://doi.org/10.1177/10983007040060020201</bibtext> </blist> <blist> <bibtext> Steinbrenner J. R., Hume K., Odom S. L., Morin K. L., Nowell S. W., Tomaszewski B., Szendrey S., McIntyre N. S., Yücesoy-Özkan S., Savage M. N. (2020). Evidence-based practices for children, youth, and young adults with Autism. National Clearinghouse on Autism Evidence and Practice Review Team, Frank Porter Graham Child Development Institute, The University of North Carolina at Chapel Hill.</bibtext> </blist> <blist> <bibtext> Suhrheinrich J. (2011). Training teachers to use pivotal response training with children with autism: Coaching as a critical component. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 34(4), 339–349. https://doi.org/10.1177/0888406411406553</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Rieth S. R., Dickson K. S., Stahmer A. C. (2020). Exploring associations between inner-context factors and implementation outcomes. Exceptional Children, 86(2), 155–173. https://doi.org/10.1177/0014402919881354</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Schetter P., England A., Melgarejo M., Nahmias A. S., Dean M., Yasuda P. (2020). Statewide interagency collaboration to support evidence-based practice scale up: The California Autism Professional Training and Information Network (CAPTAIN). Evidence-Based Practice in Child and Adolescent Mental Health, 5(4), 468–482. https://doi.org/10.1080/23794925.2020.1796545</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Stahmer A., Reed S., Schreibman L., Reisinger E. M., Mandell D. S. (2013). Implementation challenges in translating Pivotal Response Training into community settings. Journal of Autism and Developmental Disorders, 43, 2970–2976. https://doi.org/10.1007/s10803-013-1826-7</bibtext> </blist> <blist> <bibtext> Suhrheinrich J., Stahmer A. C., Schreibman L. (2007). A preliminary assessment of teachers' implementation of Pivotal Response Training. The Journal of Speech and Language Pathology—Applied Behavior Analysis, 2(1), 1–13. https://doi.org/10.1037/h0100202</bibtext> </blist> <blist> <bibtext> Tabachnick B., Fidell L. (2018). Using multivariate statistics (7th ed.). Pearson.</bibtext> </blist> <blist> <bibtext> Wong C., Odom S. L., Hume K. A., Cox A. W., Fettig A., Kucharczyk S., Brock M. E., Plavnick J. B., Fleury V. P., Schultz T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z</bibtext> </blist> </ref> <ref id="AN0155828398-28"> <title> Footnotes </title> <blist> <bibtext> The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibtext> The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this work was supported by a grant from the National Institute of Mental Health K01MH109574 (Suhrheinrich), Institute of Educational Science R324A170063 (Stahmer) and the California Department of Education.</bibtext> </blist> <blist> <bibtext> Jessica Suhrheinrich</bibtext> </blist> <blist> <bibtext>Graph</bibtext> </blist> <blist> <bibtext>https://orcid.org/0000-0003-1614-5215 Allison S Nahmias</bibtext> </blist> <blist> <bibtext>Graph</bibtext> </blist> <blist> <bibtext>https://orcid.org/0000-0003-4678-0367 Melina Melgarejo</bibtext> </blist> <blist> <bibtext>Graph</bibtext> </blist> <blist> <bibtext>https://orcid.org/0000-0002-2423-7710 Aubyn C Stahmer</bibtext> </blist> <blist> <bibtext>Graph https://orcid.org/0000-0002-1596-9848</bibtext> </blist> </ref> <aug> <p>By Jessica Suhrheinrich; Allison S Nahmias; Yue Yu; Melina Melgarejo; Patricia Schetter; Tana C Holt and Aubyn C Stahmer</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib32" firstref="ref5"></nolink> <nolink nlid="nl2" bibid="bib12" firstref="ref7"></nolink> <nolink nlid="nl3" bibid="bib18" firstref="ref8"></nolink> <nolink nlid="nl4" bibid="bib34" firstref="ref9"></nolink> <nolink nlid="nl5" bibid="bib17" firstref="ref10"></nolink> <nolink nlid="nl6" bibid="bib22" firstref="ref11"></nolink> <nolink nlid="nl7" bibid="bib33" firstref="ref12"></nolink> <nolink nlid="nl8" bibid="bib35" firstref="ref13"></nolink> <nolink nlid="nl9" bibid="bib39" firstref="ref16"></nolink> <nolink nlid="nl10" bibid="bib38" firstref="ref17"></nolink> <nolink nlid="nl11" bibid="bib10" firstref="ref18"></nolink> <nolink nlid="nl12" bibid="bib29" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib21" firstref="ref22"></nolink> <nolink nlid="nl14" bibid="bib36" firstref="ref23"></nolink> <nolink nlid="nl15" bibid="bib14" firstref="ref24"></nolink> <nolink nlid="nl16" bibid="bib13" firstref="ref25"></nolink> <nolink nlid="nl17" bibid="bib37" firstref="ref26"></nolink> <nolink nlid="nl18" bibid="bib28" firstref="ref27"></nolink> <nolink nlid="nl19" bibid="bib27" firstref="ref29"></nolink> <nolink nlid="nl20" bibid="bib31" firstref="ref30"></nolink> <nolink nlid="nl21" bibid="bib26" firstref="ref32"></nolink> <nolink nlid="nl22" bibid="bib41" firstref="ref38"></nolink> <nolink nlid="nl23" bibid="bib11" firstref="ref43"></nolink> <nolink nlid="nl24" bibid="bib23" firstref="ref48"></nolink> <nolink nlid="nl25" bibid="bib40" firstref="ref50"></nolink> <nolink nlid="nl26" bibid="bib579" firstref="ref51"></nolink> <nolink nlid="nl27" bibid="bib570" firstref="ref55"></nolink> <nolink nlid="nl28" bibid="bib555" firstref="ref56"></nolink> <nolink nlid="nl29" bibid="bib527" firstref="ref57"></nolink> <nolink nlid="nl30" bibid="bib533" firstref="ref58"></nolink> <nolink nlid="nl31" bibid="bib219" firstref="ref59"></nolink> <nolink nlid="nl32" bibid="bib222" firstref="ref60"></nolink> <nolink nlid="nl33" bibid="bib202" firstref="ref61"></nolink> <nolink nlid="nl34" bibid="bib30" firstref="ref64"></nolink> <nolink nlid="nl35" bibid="bib16" firstref="ref68"></nolink> <nolink nlid="nl36" bibid="bib24" firstref="ref70"></nolink> <nolink nlid="nl37" bibid="bib20" firstref="ref71"></nolink>
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1333832
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1333832
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Practice-Driven Research for Statewide Scale Up: Implementation Outcomes of the California Autism Professional Training and Information Network
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Suhrheinrich%2C+Jessica%22">Suhrheinrich, Jessica</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-1614-5215">0000-0003-1614-5215</externalLink>)<br /><searchLink fieldCode="AR" term="%22Nahmias%2C+Allison+S%2E%22">Nahmias, Allison S.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4678-0367">0000-0003-4678-0367</externalLink>)<br /><searchLink fieldCode="AR" term="%22Yu%2C+Yue%22">Yu, Yue</searchLink><br /><searchLink fieldCode="AR" term="%22Melgarejo%2C+Melina%22">Melgarejo, Melina</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-2423-7710">0000-0002-2423-7710</externalLink>)<br /><searchLink fieldCode="AR" term="%22Schetter%2C+Patricia%22">Schetter, Patricia</searchLink><br /><searchLink fieldCode="AR" term="%22Holt%2C+Tana+C%2E%22">Holt, Tana C.</searchLink><br /><searchLink fieldCode="AR" term="%22Stahmer%2C+Aubyn+C%2E%22">Stahmer, Aubyn C.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-1596-9848">0000-0002-1596-9848</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Autism%3A+The+International+Journal+of+Research+and+Practice%22"><i>Autism: The International Journal of Research and Practice</i></searchLink>. Apr 2022 26(3):727-736.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 10
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Institute of Mental Health (NIMH) (DHHS/NIH)<br />Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: K01MH109574<br />R324A170063
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Evidence+Based+Practice%22">Evidence Based Practice</searchLink><br /><searchLink fieldCode="DE" term="%22Autism%22">Autism</searchLink><br /><searchLink fieldCode="DE" term="%22Pervasive+Developmental+Disorders%22">Pervasive Developmental Disorders</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+Training%22">Professional Training</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Students+with+Disabilities%22">Students with Disabilities</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+Development%22">Professional Development</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Education%22">Special Education</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Schools%22">Public Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Education+Teachers%22">Special Education Teachers</searchLink><br /><searchLink fieldCode="DE" term="%22Paraprofessional+School+Personnel%22">Paraprofessional School Personnel</searchLink><br /><searchLink fieldCode="DE" term="%22School+Personnel%22">School Personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+Level%22">Knowledge Level</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Environment%22">Educational Environment</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22California%22">California</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/13623613211068224
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1362-3613<br />1461-7005
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Scaling up the use of evidence-based practice (EBP) for autism across service sectors and regions has presented a considerable challenge indicating a clear need for continued development. The California Autism Professional Training and Information Network (CAPTAIN) integrates implementation drivers into specific procedures and methodology as an implementation strategy to support statewide scale up. The current study was designed to evaluate the impact of CAPTAIN on provider-level outcomes including attitude toward, and knowledge, fidelity and use of autism EBPs, and overall classroom quality. Overall, results indicated variability across measures, with some significant differences between CAPTAIN-trained and non-CAPTAIN-trained providers. CAPTAIN-trained providers reported more openness to EBP. Significantly more CAPTAIN-trained direct service providers reported collecting fidelity of implementation data (X[superscript 2](2, N = 1515) = 10.95, p = 0.004), collecting student data (X[superscript 2](2, N = 1509) = 14.19, p = 0.001), and reported using their primary EBP with "most or all students" (X[superscript 2](2, N = 1514) = 11.41, p = 0.003) than providers not trained by CAPTAIN. In summary, these preliminary findings show promise for the efficacy of the CAPTAIN model to increase dissemination and implementation of EBP at the classroom level.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1333832
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1333832
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/13623613211068224
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
        StartPage: 727
    Subjects:
      – SubjectFull: Evidence Based Practice
        Type: general
      – SubjectFull: Autism
        Type: general
      – SubjectFull: Pervasive Developmental Disorders
        Type: general
      – SubjectFull: Professional Training
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
      – SubjectFull: Students with Disabilities
        Type: general
      – SubjectFull: Professional Development
        Type: general
      – SubjectFull: Special Education
        Type: general
      – SubjectFull: Public Schools
        Type: general
      – SubjectFull: Special Education Teachers
        Type: general
      – SubjectFull: Paraprofessional School Personnel
        Type: general
      – SubjectFull: School Personnel
        Type: general
      – SubjectFull: Knowledge Level
        Type: general
      – SubjectFull: Educational Environment
        Type: general
      – SubjectFull: California
        Type: general
    Titles:
      – TitleFull: Practice-Driven Research for Statewide Scale Up: Implementation Outcomes of the California Autism Professional Training and Information Network
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Suhrheinrich, Jessica
      – PersonEntity:
          Name:
            NameFull: Nahmias, Allison S.
      – PersonEntity:
          Name:
            NameFull: Yu, Yue
      – PersonEntity:
          Name:
            NameFull: Melgarejo, Melina
      – PersonEntity:
          Name:
            NameFull: Schetter, Patricia
      – PersonEntity:
          Name:
            NameFull: Holt, Tana C.
      – PersonEntity:
          Name:
            NameFull: Stahmer, Aubyn C.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-print
              Value: 1362-3613
            – Type: issn-electronic
              Value: 1461-7005
          Numbering:
            – Type: volume
              Value: 26
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Autism: The International Journal of Research and Practice
              Type: main
ResultId 1