View in EDS HTML Full Text PDF Full Text

Treatment Fidelity: What It Is and Why It Matters

Saved in:
Bibliographic Details
Title: Treatment Fidelity: What It Is and Why It Matters
Language: English
Authors: Sanetti, Lisa M. H. (ORCID 0000-0002-4480-9886), Cook, Bryan G. (ORCID 0000-0001-9294-0873), Cook, Lysandra
Source: Learning Disabilities Research & Practice. Feb 2021 36(1):5-11.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 7
Publication Date: 2021
Document Type: Journal Articles
Reports - Descriptive
Descriptors: Fidelity, Intervention, Program Implementation, Data Interpretation
DOI: 10.1111/ldrp.12238
ISSN: 0938-8982
Abstract: Treatment fidelity refers to the extent to which an intervention is implemented as planned. If researchers do not assess and report treatment fidelity, or if treatment fidelity is shown to be low, findings from intervention studies are difficult to interpret, because the intervention may not have been implemented as planned. In this article, our aim is to inform research consumers by discussing treatment fidelity and its primary dimensions, providing guidelines for interpreting treatment fidelity, considering implications of treatment fidelity for research and practice, and illustrating how fidelity is reported in two recent studies. Our take-home message is that when one is interpreting intervention studies, it is important to consider whether the interventions were applied as planned, or with fidelity.
Abstractor: As Provided
Entry Date: 2021
Accession Number: EJ1287879
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFG-dsIJnwNuhNS6LtOsNb8AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDL-zyrlkRKBNZ_YwxgIBEICBmxFGDqW_NbK1WavzgiuGiDURUTI_bF-vEgZU207A_lQrfRv668SwO2pCFtSRveQYQH0gRiSjAr5BzosD8cjcZiIQEKVtXDDiXuoRF9mVFAjWx601-RHwh3IhqZdKXTdrawJahph-hhMXpK7tHMYn1msCLB0Zd_degTdOyWt2b1a54k9FfQK_EQd8NHJtD0A9P7tXGfUfkoIG7F3a
Text:
  Availability: 1
  Value: <anid>AN0148996736;7mj01feb.21;2021Mar03.07:31;v2.2.500</anid> <title id="AN0148996736-1">Treatment Fidelity: What It Is and Why It Matters </title> <p>Treatment fidelity refers to the extent to which an intervention is implemented as planned. If researchers do not assess and report treatment fidelity, or if treatment fidelity is shown to be low, findings from intervention studies are difficult to interpret, because the intervention may not have been implemented as planned. In this article, our aim is to inform research consumers by discussing treatment fidelity and its primary dimensions, providing guidelines for interpreting treatment fidelity, considering implications of treatment fidelity for research and practice, and illustrating how fidelity is reported in two recent studies. Our take‐home message is that when one is interpreting intervention studies, it is important to consider whether the interventions were applied as planned, or with fidelity.</p> <p> <emph>Mr. Cooper and Ms. Romero cotaught a 3rd‐grade classroom at Wilson Elementary. Six of their students, three of whom had learning disabilities, struggled with reading fluency and made many errors when reading grade‐level content. Mr. Cooper, the special education teacher, remembered learning about repeated reading in his teacher preparation program and thought it would be perfect for him to implement during RTI (response to intervention) block with these students. He searched and found research examining the effectiveness of repeated reading, and was pleased to find that multiple studies showed it was effective for improving the reading fluency of elementary students with and at risk for learning disabilities (see Lee & Yoon</emph>, 2017<emph>, for a review of this literature). He remembered that repeated reading involves students reading the same passages repeatedly until fluency, which he defined as achieving at least 95% accuracy, is attained. During RTI block, Mr. Cooper had students read grade‐level passages multiple times until they felt they had mastered it, at which time he assessed their accuracy. He monitored student progress by having students read different grade‐level passages, and then recording and graphing their accuracy. After two months of implementing repeated reading, Mr. Cooper was disappointed that most of his students were making minimal progress and falling further behind their peers. During their planning period, Ms. Romero asked Mr. Cooper if he was measuring treatment fidelity of repeated reading to make sure he was implementing it in the same way as in the research studies showing it to be effective. "What is treatment fidelity?" responded Mr. Cooper</emph>.</p> <p>This article is part of the research‐to‐practice special series in <emph>Learning Disabilities Research & Practice</emph>, a series that aims to provide accessible information on core research concepts in order to facilitate practitioners and other education stakeholders being informed and critical consumers of the research literature. The purpose of this article is to provide an overview of treatment fidelity (also referred to as treatment integrity, intervention integrity, and procedural reliability), or the extent to which an intervention is implemented as planned, and to discuss its importance for interpreting intervention research. Toward that end, in this article we (a) describe treatment fidelity dimensions, assessment, and supports; (b) discuss the relevance of treatment fidelity for research and practice; and (c) provide examples of how treatment fidelity is examined and reported in recent research involving students with LD. Our take‐home message is that when one is interpreting intervention studies, it is important to consider whether the interventions being studied were applied as planned, or with fidelity.</p> <p>Traditionally, it was assumed that interventions, including those implemented with the aim of improving outcomes for students with learning disabilities (LD), are implemented as designed in research studies. Yet the reality is often otherwise (Sanetti & Collier‐Meek, 2019). Significantly, when interventions are not implemented as planned (i.e., without fidelity), learner outcomes are less likely to improve (Scott, Gage, Hirn, Lingo, & Burt, 2019). In fact, if treatment fidelity is not assessed, or is documented as being low in an intervention study, it is impossible to know whether the intervention was effective or not, regardless of study results. In other words, the effects of an intervention cannot be examined meaningfully if it is not clear that the intervention was implemented appropriately. Treatment fidelity also has important implications for practice. When educators implement an evidence‐based intervention in their classes, implementing the practice with fidelity is critical, as students may not respond positively to an intervention they did not receive as intended. As applied to both research and practice, treatment fidelity is an important, multidimensional construct that can be challenging to consider fully.</p> <hd id="AN0148996736-2">TREATMENT FIDELITY</hd> <p></p> <hd id="AN0148996736-3">Dimensions of Treatment of Fidelity</hd> <p>As the understanding of treatment fidelity has evolved, new conceptual models and data suggest it is a complex and multidimensional construct (Sanetti & Fallon, 2011). Treatment fidelity is recognized to have content‐, quantity‐, and process‐related dimensions. The content dimension, referred to as adherence, is the extent to which intervention steps were implemented as planned. The quantity dimension has two related components: dosage and exposure. Dosage is the frequency with which, and duration for which, an intervention was delivered. Exposure is the frequency with which, and duration for which, a recipient (e.g., student) received the intervention. The process dimension, often referred to as quality, is how well intervention steps were implemented. Each of these four dimensions provides unique information about intervention implementation, and consideration of multiple dimensions can result in different determinations about the adequacy of treatment fidelity. See Table 1 for an overview of these dimensions of treatment fidelity.</p> <p>1 TABLEKey Terms Related to Treatment Fidelity Dimensions and Methods of Assessment</p> <p> <ephtml> <table><thead><tr><th>Term</th><th align="center">Definition</th><th align="center">Possible Methods of Assessment</th><th align="center">Example</th></tr></thead><tbody><tr><td>Adherence</td><td>The extent to which the intervention steps were implemented as planned</td><td>Direct observation, permanent product, self‐report</td><td>Of the 10 steps to the "Best Practice" intervention, a teacher implemented 8 on her first day, resulting in an adherence level of 8/10, or 80%. Intervention step‐level data across the first two weeks showed that when she implemented the intervention, she skipped the same 2 intervention steps and fully implemented 8 each day.</td></tr><tr><td>Dosage</td><td>The frequency with and duration for which an intervention was delivered</td><td>Direct observation, permanent product, self‐report</td><td>The "Best Practice" intervention is supposed to be implemented 3 times per week for 20 minutes. During the first week, the teacher implemented the intervention once for 15 minutes, resulting in 33% of the expected weekly frequency; 0%, 0%, and 75% of the expected daily duration; and 25% of the weekly duration.</td></tr><tr><td>Exposure</td><td>The frequency with and duration for which a recipient received the intervention</td><td>Direct observation, permanent product, self‐report</td><td>Three students are in this "Best Practice" small‐group intervention. Of the 20 implemented sessions, Student A has missed 0 sessions, Student B has missed 2 sessions, and Student C has missed 10 sessions. This results in exposure estimates of 100%, 90%, and 50%, respectively.</td></tr><tr><td>Quality</td><td>How well intervention steps were implemented</td><td>Direct observation, self‐report</td><td>Of the 8 "Best Practice" intervention steps the teacher implemented, she did so with a high level of quality, earning 95% of possible rating points. On days when she did not implement the intervention (0% adherence, dosage, and exposure), quality could not be rated.</td></tr></tbody></table> </ephtml> </p> <p>Adherence data provide information about both the extent to which an intervention is implemented each day (often quantified as a percentage of intervention steps implemented) and the consistency with which each intervention step is implemented over time (often quantified as a percentage of days a step was implemented). These two types of adherence provide unique information, as an educator may, for example, consistently demonstrate adequate daily adherence (e.g., implementing 90% of intervention steps daily), but may consistently fail to implement a specific element of the intervention (e.g., providing clear directions on 0% of days). Adherence is typically considered a foundational dimension of treatment fidelity, in that if the intervention steps are not implemented, then the other dimensions may be less important or even irrelevant.</p> <p>Dosage provides information about the frequency with which, and duration for which, an intervention is implemented. Given that time is an important barrier to fidelity, this dimension of treatment fidelity is highly relevant. For example, it is possible that an educator demonstrates a high level of adherence, but only implements an intervention (a) for 10 minutes daily, instead of the recommended 20 minutes; or (b) twice weekly instead of daily. In these instances, it is possible that only looking at adherence data may lead one to believe that implementation is adequate; evaluating both adherence and dosage data, however, might make it clear that students are not truly receiving the intervention as planned. Exposure refers to the degree to which students actually receive the intervention, regardless of the time it is provided. For example, it is possible that an educator implements an intervention with a high level of adherence and dosage, but that the student is regularly tardy or absent; and thus, the student's exposure to the intervention is not at the planned level, which will hinder intervention effectiveness.</p> <p>Finally, process, or quality, refers to how well an intervention is implemented. Given the nature of most school‐based interventions, quality often includes interpersonal interaction (e.g., providing positive feedback in a timely and enthusiastic way), sensitivity to students' needs (e.g., adjusting the pace of delivery), and effective management of materials (e.g., having the right manipulatives readily available; Sanetti & Fallon, 2011). It is possible for there to be high levels of adherence, dosage, and exposure, but for the quality of the intervention delivery to be low. Low quality has the potential to negatively impact recipient engagement and responsiveness.</p> <p>Research evidence suggests that evaluating different fidelity dimension data can (a) predict student outcomes (e.g., high fidelity across dimensions predicts higher student outcomes; Domitrovich, Gest, Jones, Gill, & DeRousie, 2010), (b) influence decisions about intervention effectiveness (Sanetti & Fallon, 2011), and (c) inform supports for improving fidelity levels (Sanetti & Collier‐Meek, 2019). Therefore, it is important that researchers assess and report multiple types of treatment fidelity when investigating the effects of an intervention.</p> <hd id="AN0148996736-4">Assessment of Treatment Fidelity</hd> <p>Despite decades of research on treatment fidelity in education, the literature on treatment fidelity assessment is in its infancy. Although researchers occasionally use other methods (e.g., permanent product review, implementer self‐report), the vast majority use direct observation to assess treatment fidelity. Direct observation involves observing an intervention in person or via video, and rating the implementation of intervention steps. Direct observation is considered to be the "gold standard" in fidelity assessment, and it can allow for collection of adherence, dosage, exposure, and quality. Research results indicate that direct‐observation fidelity data are the most resource‐intensive to collect, but also that these data can be reliable, accurate, and predictive of student outcomes (Dart, Collier‐Meek, Chambers, & Murphy, 2020).</p> <p>For reported fidelity assessment data to be useful, it is important to know how to interpret them. Available guidelines suggest that 80–100% adherence is considered high, 50–79% is considered moderate, and 0–49% is considered low treatment fidelity (Perepletchikova & Kazdin, 2005). The adequacy of dosage is less straightforward, and varies based on the intervention. For example, some interventions are designed to be implemented for an exact number and duration of sessions (e.g., daily for 15 minutes), whereas others provide flexibility (e.g., 30 minutes within a week), and some use performance‐based criteria (e.g., repeat sessions until student reaches 90% correct). As a result, one needs to interpret dosage in comparison to the researcher's description of the intervention as planned. With regard to exposure, students will not benefit from interventions they do not receive. Chronic absenteeism and truancy data consistently indicate that exposure levels less than 90% are predictive of poorer student outcomes across academic, behavior, and social–emotional domains (Smerillo, Reynolds, Temple, & Ou, 2018). Quality is often assessed by observers using rating scales (e.g., 0 = little or no feedback provided, materials not readily available; 1 = some feedback provided, some materials readily available; and 2 = frequent feedback provided in a timely and enthusiastic way, all materials readily available). Guidelines specific to interpreting quality estimates are not yet available; typically, the guidelines for adherence are used for quality as well. When one is interpreting quality data, it is important to pay attention to how quality estimates are calculated. Most often, only those intervention steps that are implemented are rated with regard to the quality of their implementation, which can result in inflated quality estimates. Thus, one should be cautious about a study reporting 90% quality but only 50% adherence, for example. In this case, although the elements of the intervention that were implemented were applied with quality, half of the elements were not implemented at all. The interpretation of treatment fidelity estimates has been, and continues to be, an area of debate in the field. Guidelines for evaluating will continue to evolve as more research becomes available.</p> <p>As displayed in Figure 1, educators can analyze student outcome data and treatment fidelity together to interpret findings of intervention research. There are four possible combinations of student‐outcome and treatment‐fidelity data. The first is when student outcomes improve meaningfully and treatment fidelity data are adequate. When this combination is reported in a study, it is reasonable to conclude that the intervention, which was implemented as planned, was effective for study participants. When an intervention study reports that student outcomes improve meaningfully but treatment fidelity is inadequate (i.e., low or unreported), although readers know that something occurred that resulted in improved student performance, there is no evidence indicating that the intervention was implemented as planned. Therefore, it cannot be concluded that the planned intervention was the cause of the improved outcomes. This finding does not necessarily mean that the intervention is ineffective—only that clear conclusions about its effectiveness cannot be drawn from the study. When an intervention study reports that student outcomes do not improve meaningfully, but that treatment fidelity is adequate, readers can conclude that the intervention was ineffective for study participants. Finally, when student outcomes do not improve and treatment fidelity is inadequate, readers should not conclude that the intervention was ineffective. Whatever the researchers implemented did not improve student outcomes, but—because treatment fidelity is inadequate—no conclusions can be reached regarding intervention effectiveness.</p> <p> <img src="https://imageserver.ebscohost.com/img/embimages/rdk/7MJ/01feb21/ldrp12238-fig-0001.jpg?ephost1=dGJyMNXb4kSepq84yOvqOLCmsE6epq5Srqa4SK6WxWXS" alt="ldrp12238-fig-0001.jpg" title="1 Interpreting treatment fidelity and student outcome data." /> </p> <p></p> <hd id="AN0148996736-6">TREATMENT FIDELITY IN RESEARCH</hd> <p>Perhaps the greatest relevance of treatment fidelity for research is related to the validity, or accuracy, of study findings. Treatment fidelity data are essential to drawing valid conclusions about the effects of an intervention on dependent variables (e.g., student outcomes; see Figure 1). For instance, without treatment fidelity data, if students show little to no improvement at the conclusion of an intervention study, the researchers may conclude that the intervention is ineffective, when in fact it may be that the intervention would have been highly effective if it had been implemented as planned. Not measuring and reporting treatment fidelity also threatens the degree to which study findings apply in other settings and with other students (i.e., external validity). For example, if a researcher reports positive effects resulting from an intervention in their study without examining treatment fidelity, it is possible that the intervention was implemented differently than was planned, and that those differences may have caused improved student outcomes. Thus, when other researchers and educators implement the intervention as planned, they may not obtain the same positive effects reported in the study.</p> <p>These validity issues highlight the importance of assessing and reporting fidelity data in intervention evaluation research. Reviews show that reporting of treatment fidelity data has increased over time in the special education research literature. Only about 20% of researchers assessed and reported quantitative fidelity data from the 1960s to 1990s, but this percentage has increased to 68% (Swanson, Wanzek, Haring, Ciullo, & McCully, 2019). Despite this improvement, it is important for researchers in special education (a) to continue to increase the rate at which treatment fidelity is assessed and reported in intervention studies, and (b) to conduct additional research to inform clear guidelines regarding minimally acceptable levels of treatment fidelity.</p> <p>Treatment fidelity data are essential to drawing valid conclusions about intervention effectiveness in research. Rates of researchers reporting fidelity data are growing over time, perhaps because the importance of fidelity assessment is now widely accepted. Treatment fidelity also has important and related implications when implementing research‐based interventions in practice.</p> <hd id="AN0148996736-7">TREATMENT FIDELITY IN PRACTICE</hd> <p>As with its role in research, evaluation of treatment fidelity is critical to drawing valid conclusions about intervention effectiveness in practice. The current reality is that decades of research and billions of dollars devoted to developing evidence‐based interventions to promote improved student outcomes are not achieving their potential impact, in part due to inadequate implementation in schools. For example, poor fidelity frequently is evident across each tier in multitiered systems of support (MTSS), as well as in special education, and treatment fidelity has been noted as constituting the biggest hurdle to reaching the full potential of MTSS (Noell & Gansle, 2006). Treatment fidelity may be especially critical when teaching students with disabilities, as students with disabilities may be less tolerant of deviations from fidelity than other learners.</p> <p>Despite the central nature of treatment fidelity data to intervention evaluation and success in MTSS and special education, it is rarely collected in schools (Cochrane, Sanetti, & Minster, 2019). In the absence of treatment fidelity data, however, if student performance does not increase, educators do not know whether that is because the intervention did not work for the student or because the teacher did not implement the intervention appropriately. When examining and interpreting treatment fidelity data in practice, it is essential to (a) recognize the difference between adaptation and drift; (b) understand what elements of an intervention can be adapted without rendering the intervention ineffective; and (c) document adaptations proactively and holistically, and ensure they are reflected on fidelity measures.</p> <p>Program drift refers to unplanned deviations from the intervention, whereas adaptation refers to purposeful changes to the content or method of intervention delivery. Program drift is common and often occurs unconsciously. Yet because program drift changes critical elements of the intervention, it can lessen the intervention's positive effects. For example, a teacher may implement Incremental Rehearsal with a student who is struggling with their math facts. Initially, the teacher provides practice on an unknown fact, interspersed with practice on increasing numbers of known facts, as recommended by the research (Burns, 2005). Over time, however, the teacher provides fewer and fewer known facts when the student answers incorrectly and then moves on to the next unknown. This drift decreases behavioral momentum, the basis for the intervention. Program drift can be identified by collecting fidelity data that will indicate how implementation is deviating from the planned intervention.</p> <p>Adaptation may occur before or during implementation and, when done thoughtfully, can improve the effects of the intervention. In general, critical elements of interventions that are central to the theoretical underpinnings of the intervention, or that have been empirically shown to predict desired outcomes, should not be adapted. Other elements of an intervention, however, can be adapted to address unique learning needs of students, or to enhance alignment with teacher expertise and resources. For example, a teacher may decide to implement Cover, Copy, Compare to increase a student's acquisition of math facts. After a few days of implementation, the teacher sees that the student's poor fine motor skills are causing frustration during the intervention. The teacher is concerned that the student soon will not want to engage in the intervention much longer. To address this issue, the teacher purposefully adapts the intervention so that the student responds to the math problems orally while the teacher writes them down. This planned adaptation maintains the critical components of the intervention and increases the student's willingness to engage in the intervention.</p> <hd id="AN0148996736-8">EXAMPLES OF TREATMENT FIDELITY IN RECENT RESEARCH</hd> <p>In this section, we discuss two examples of recent intervention studies evaluating the effects of an intervention on the outcomes of students with and at risk for LD, the first a group experiment and the second a single‐case design, highlighting how the authors assessed and reported treatment fidelity. Wanzek et al. (2020) conducted an experiment to examine the effects of two different interventions: (a) an explicit, phonics‐based reading intervention, the <emph>Lindamood Phoneme Sequencing Program</emph> (<emph>LiPS</emph>; Lindamood & Lindamood, 2011); and (b) <emph>LiPS</emph> plus a positive‐growth‐mindset intervention, Brainology® (Mindset Works, 2016). Three‐hundred and sixty‐one 4th‐grade students with reading difficulties (14% of whom had a disability) were randomly assigned to either a business‐as‐usual control condition, the reading‐only intervention, or the reading + mindset intervention. The reading intervention was designed to be implemented daily for 45 minutes per lesson, whereas the mindset intervention was designed to be implemented twice a week for 30 minutes per lesson.</p> <p>Regarding exposure, the authors reported that students in the two intervention conditions received 73.5 reading lessons, on average, while all students in the reading + mindset condition received all 24 mindset lessons. The average length of the reading lessons was 43 minutes; average length of mindset lessons was not reported. Adherence was measured by trained observers who observed lessons of each reading interventionist monthly, and each trained mindset interventionist five separate times. Observers rated the implementation of (a) critical elements of the reading intervention on a 0 (not completed) to 3 (excellent implementation) scale, and (b) activities in the mindset intervention on a 0 (1/3 or less of the activity completed) to 2 (2/3 or more of the activity completed) scale. Average adherence ratings were high: 2.7 (out of 3, or 90%) for the reading intervention, and 1.8 (out of 2, or 90%) for the mindset intervention.</p> <p>Overall quality of individualization, guided practice, pacing/wait time, monitoring, explicit and specific feedback, time management, and behavior management in observed reading lessons were also rated on a 0–2 scale. The average quality rating was 2.73 (>90%), indicating generally high‐quality implementation of the reading intervention. Overall quality of the mindset intervention was not examined. Student engagement in both reading and mindset lessons was also rated on a 0–2 scale. Results indicated high levels of student engagement, with average ratings of 2.82 and 2.70 for the reading and mindset interventions, respectively (≥90%). These indications of adequate treatment fidelity support the validity of study findings that (a) students in both intervention conditions improved significantly more than those in the control condition on nonword reading, (b) students in the reading‐only condition improved significantly more than the control group on phonological processing, and (c) students in the reading‐only condition improved significantly more than those in the reading + mindfulness condition on reading fluency. No differences were found among the three groups on word reading, reading comprehension, or mindset (Wanzek et al., 2020).</p> <p>Ok and Bryant (2016) used a single‐case (multiple‐probe across participants) design to evaluate the effects of an intervention that involved explicit instruction on a doubling strategy and independent practice using a tablet‐based game on multiplication‐fact fluency for four 5th‐grade students with LD. Results of the study showed a functional relation between the intervention and students' performance on daily progress‐monitoring probes measuring rate of correct responses on multiplication facts, with improved performance maintaining after the intervention.</p> <p>The authors examined adherence using a 20‐item checklist that measured critical elements of the scripted intervention sessions such as making materials available, modeling procedures, providing guided practice, providing independent practice, providing feedback and support, and graphing daily data. Each item was rated on a 3‐point scale (0 = behavior not observed, 1 = inconsistent implementation, 2 = high level of implementation). Four (of 15 total) sessions were observed for each participant. Adherence was high, with 98% of possible ratings points earned across sessions observed. Although dosage and exposure were not reported, dosage can be inferred from the graphs reported in most single‐case design studies. For example, Ok and Bryant (2016) reported that each student was to receive 15 intervention sessions. Three of the four students had 15 intervention data points, derived from the progress‐monitoring probes administered at the end of each intervention session, on their graphs, indicating full dosage of the intervention sessions for those students. The remaining student had 14 intervention data points, indicating that one intervention session was not provided. Although procedural quality was not assessed, the adherence and dosage data support adequate treatment fidelity of the intervention.</p> <hd id="AN0148996736-9">CONCLUSION</hd> <p> <emph>Ms. Romero explained to Mr. Cooper that treatment fidelity means the degree to which an intervention is implemented as planned. She suggested she assess Mr. Cooper's adherence for repeated reading. Ms. Romero developed a checklist of the critical elements for repeated reading from articles on the intervention. She then observed Mr. Cooper teaching repeated reading for four consecutive days. On average, Mr. Cooper's adherence was high; he implemented 90% of the intervention elements each day. Mr. Cooper, however, failed to provide corrective feedback on the students' reading, which is a critical element of repeated reading (Therrien & Kubina</emph>, 2006<emph>), on any of the days. Ms. Romero and Mr. Cooper discussed the treatment fidelity assessments. Mr. Cooper had forgotten that corrective feedback was a critical part of repeated reading. He began to include that element of the intervention when he taught, and student reading fluency soon began to improve. Ms. Romero periodically observed Mr. Cooper teaching repeated reading throughout the rest of the year to make sure that program drift did not occur</emph>.</p> <p>Treatment fidelity, or the extent to which an intervention is implemented as planned, is—as Mr. Cooper found—an important construct for educators to consider when implementing research‐based practices. Similarly, as our take‐home message indicates, when one is interpreting intervention research, it is important to consider whether the intervention being studied was applied as planned, or with fidelity. If researchers did not assess fidelity, or if fidelity is shown to be low, it is difficult to make sense of the outcomes of intervention studies because it is not clear how the intervention was delivered. With the goal of providing readers with an initial understanding of treatment fidelity to better interpret and apply research findings, in this article we discussed the multiple dimensions of treatment fidelity (adherence, dosage, exposure, and quality), provided guidelines for interpreting reports of treatment fidelity in research articles, considered implications of treatment fidelity for research and practice, and examined how treatment fidelity was reported in two recent research articles in the special education literature.</p> <ref id="AN0148996736-10"> <title> REFERENCES </title> <blist> <bibl id="bib1" type="bt">1</bibl> <bibtext> Burns, M. K. (2005). Using incremental rehearsal to increase fluency of single digit multiplication facts with children identified as learning disabled in mathematics computation. Education and Treatment of Children, 28, 237 – 249. https://<ulink href="http://www.jstor.org/stable/42899847">www.jstor.org/stable/42899847</ulink></bibtext> </blist> <blist> <bibl id="bib2" type="bt">2</bibl> <bibtext> Cochrane, W. S., Sanetti, L. M. H., & Minster, M. C. (2019). School psychologists' beliefs and practices about treatment integrity in 2008 and 2017. Psychology in the Schools, 56, 295 – 306. https://doi.org/10.1002/pits.22177</bibtext> </blist> <blist> <bibl id="bib3" type="bt">3</bibl> <bibtext> Dart, E. H., Collier‐Meek, M. A., Chambers, C., & Murphy, A. (2020). Multi‐informant assessment of treatment integrity in the classroom. Psychology in the Schools, 57 (5), 805 – 822. https://doi.org/10.1002/pits.22351</bibtext> </blist> <blist> <bibl id="bib4" type="bt">4</bibl> <bibtext> Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & DeRousie, R. M. S. (2010). Implementation quality: Lessons learned in the context of the head start REDI trial. Early Childhood Research Quarterly, 25, 284 – 298. https://doi.org/10.1016/j.ecresq.2010.04.001</bibtext> </blist> <blist> <bibl id="bib5" type="bt">5</bibl> <bibtext> Lee, J., & Yoon, S. Y. (2017). The effects of repeated reading on reading fluency for students with reading disabilities: A meta‐analysis. Journal of Learning Disabilities, 50 (2), 213 – 224. https://doi.org/10.1177/0022219415605194</bibtext> </blist> <blist> <bibl id="bib6" type="bt">6</bibl> <bibtext> Lindamood, P. C., & Lindamood, P. D. (2011). LiPS: The Lindamood phoneme sequencing® program for reading, spelling, and speech, 4th edn. PRO‐ED : Austin, TX. https://<ulink href="http://www.proedinc.com/Products/13600/lips-the-lindamood-phoneme-sequencing-program-for-reading-spelling-and-speechfourth-edition-complete-kit.aspx">www.proedinc.com/Products/13600/lips-the-lindamood-phoneme-sequencing-program-for-reading-spelling-and-speechfourth-edition-complete-kit.aspx</ulink></bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> Mindset Works. (2016). The Brainology curriculum elementary guide to implementation. Retrieved from https://<ulink href="http://www.mindsetworks.com">www.mindsetworks.com</ulink></bibtext> </blist> <blist> <bibl id="bib8" type="bt">8</bibl> <bibtext> Noell, G. H., & Gansle, K. A. (2006). Assuring the form has substance: Treatment plan implementation as the foundation of assessing response to intervention. Assessment for Effective Intervention, 32 (1), 32 – 39. https://doi.org/10.1177/15345084060320010501</bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext> Ok, M. W., & Bryant, D. P. (2016). Effects of a strategic intervention with iPad practice on the multiplication fact performance of fifth‐grade students with learning disabilities. Learning Disability Quarterly, 39 (3), 146 – 158. https://doi.org/10.1177/0731948715598285</bibtext> </blist> <blist> <bibtext> Perepletchikova, F., & Kazdin, A. E. (2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365 – 383. https://doi.org/10.1093/clipsy.bpi045</bibtext> </blist> <blist> <bibtext> Sanetti, L. M. H., & Collier‐Meek, M. C. (2019). Supporting successful interventions in schools: Tools to plan, evaluate, and sustain effective implementation. Guilford Press : New York, NY. https://<ulink href="http://www.guilford.com/books/Supporting-Successful-Interventions-in-Schools/Sanetti-Collier-Meek/9781462537730">www.guilford.com/books/Supporting-Successful-Interventions-in-Schools/Sanetti-Collier-Meek/9781462537730</ulink></bibtext> </blist> <blist> <bibtext> Sanetti, L. M. H., & Fallon, L. M. (2011). Treatment integrity assessment: How estimates of adherence, quality, and exposure influence interpretation of implementation. Journal of Educational and Psychological Consultation, 21, 209 – 232. https://doi.org/10.1080/10474412.2011.595163</bibtext> </blist> <blist> <bibtext> Scott, T. M., Gage, N. A., Hirn, R. G., Lingo, A. S., & Burt, J. (2019). An examination of the association between MTSS implementation fidelity measures and student outcomes. Preventing School Failure, 63, 308 – 316. https://doi.org/10.1080/1045988X.2019.1605971</bibtext> </blist> <blist> <bibtext> Smerillo, N. E., Reynolds, A. J., Temple, J. A., & Ou, S. R. (2018). Chronic absence, eighth‐grade achievement, and high school attainment in the Chicago Longitudinal Study. Journal of School Psychology, 67, 163 – 178. https://doi.org/10.1016/j.jsp.2017.11.001</bibtext> </blist> <blist> <bibtext> Swanson, E., Wanzek, J., Haring, C., Ciullo, S., & McCully, L. (2019) Intervention fidelity in special and special education research journals. Journal of Special Education, 47, 3 – 13. https://doi.org/10.1177/0022466911419516</bibtext> </blist> <blist> <bibtext> Therrien, W. J., & Kubina Jr, R. M. (2006). Developing reading fluency with repeated reading. Intervention in School and Clinic, 41 (3), 156 – 160. https://doi.org/10.1177/10534512060410030501</bibtext> </blist> <blist> <bibtext> Wanzek, J., Otaiba, S. A., Petscher, Y., Lemons, C. J., Gesel, S. A., Fluhler, S., ... Rivas, B. K. (2020). Comparing the effects of reading intervention versus reading and mindset intervention for upper elementary students with reading difficulties. Journal of Learning Disabilities. Advance online publication. https://doi.org/10.1177/00222194209</bibtext> </blist> </ref> <aug> <p>By Lisa M. H. Sanetti; Bryan G. Cook and Lysandra Cook</p> <p>Reported by Author; Author; Author</p> <p></p> <p>Lisa M. H. Sanetti is a professor in the School Psychology Program at the University of Connecticut Neag School of Education. She received her PhD in educational psychology from the University of Wisconsin‐Madison. Her primary research interests are implementation science, educator well‐being, and school‐based mental health.</p> <p>Bryan G. Cook is a professor in the Special Education Program at the University of Virginia School of Education and Human Development, and received his PhD in special education at the University of California at Santa Barbara. His primary lines of inquiry include open science, conducting meta‐research on the special education research base, and evidence‐based practices.</p> <p>Lysandra Cook is an associate professor in the Special Education Program at the University of Virginia School of Education and Human Development, and received her PhD from Kent State University. Her scholarly interests revolve around translating research to practice in special education, including (a) identifying and implementing evidence‐based practices, (b) supporting pre‐ and in‐service teachers to be critical consumers of research, and (c) researching high‐quality teacher preparation.</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1287879
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Treatment Fidelity: What It Is and Why It Matters
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Sanetti%2C+Lisa+M%2E+H%2E%22">Sanetti, Lisa M. H.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4480-9886">0000-0002-4480-9886</externalLink>)<br /><searchLink fieldCode="AR" term="%22Cook%2C+Bryan+G%2E%22">Cook, Bryan G.</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-9294-0873">0000-0001-9294-0873</externalLink>)<br /><searchLink fieldCode="AR" term="%22Cook%2C+Lysandra%22">Cook, Lysandra</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Learning+Disabilities+Research+%26+Practice%22"><i>Learning Disabilities Research & Practice</i></searchLink>. Feb 2021 36(1):5-11.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 7
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2021
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Descriptive
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Fidelity%22">Fidelity</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Implementation%22">Program Implementation</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Interpretation%22">Data Interpretation</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/ldrp.12238
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0938-8982
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Treatment fidelity refers to the extent to which an intervention is implemented as planned. If researchers do not assess and report treatment fidelity, or if treatment fidelity is shown to be low, findings from intervention studies are difficult to interpret, because the intervention may not have been implemented as planned. In this article, our aim is to inform research consumers by discussing treatment fidelity and its primary dimensions, providing guidelines for interpreting treatment fidelity, considering implications of treatment fidelity for research and practice, and illustrating how fidelity is reported in two recent studies. Our take-home message is that when one is interpreting intervention studies, it is important to consider whether the interventions were applied as planned, or with fidelity.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2021
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1287879
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1287879
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/ldrp.12238
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
        StartPage: 5
    Subjects:
      – SubjectFull: Fidelity
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Program Implementation
        Type: general
      – SubjectFull: Data Interpretation
        Type: general
    Titles:
      – TitleFull: Treatment Fidelity: What It Is and Why It Matters
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Sanetti, Lisa M. H.
      – PersonEntity:
          Name:
            NameFull: Cook, Bryan G.
      – PersonEntity:
          Name:
            NameFull: Cook, Lysandra
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 02
              Type: published
              Y: 2021
          Identifiers:
            – Type: issn-print
              Value: 0938-8982
          Numbering:
            – Type: volume
              Value: 36
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Learning Disabilities Research & Practice
              Type: main
ResultId 1