View in EDS HTML Full Text PDF Full Text

Conceptualizing Fidelity of Implementation of a Technology-Delivered Self-Determination Intervention

Saved in:
Bibliographic Details
Title: Conceptualizing Fidelity of Implementation of a Technology-Delivered Self-Determination Intervention
Language: English
Authors: Karrie A. Shogren, Valerie L. Mazzotti, Daria Gerasimova, Darcy Fredrick, Stephen M. Kwiatek, Sheida K. Raley, Helena Rodrigues Ferreira, Aaron M. Beuoy, Tyler A. Hicks
Source: Inclusion. 2025 13(2):116-132.
Availability: American Association on Intellectual and Developmental Disabilities. P.O. Box 1897, Lawrence, KS 66044-1897. Tel: 785-843-1235; Fax: 785-843-1274; e-mail: AJMR@allenpress.com; Web site: https://meridian.allenpress.com/inclusion
Peer Reviewed: Y
Page Count: 17
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Descriptors: Technology Uses in Education, Students with Disabilities, Inclusion, Fidelity, Goal Orientation, Self Determination, Standard Setting, Program Effectiveness
DOI: 10.1352/2326-6988-13.2.116
ISSN: 2326-6988
Abstract: As technology becomes a more central support for students with disabilities in inclusive schools, there is a need to understand fidelity of implementation of technology-delivered interventions. We propose a framework for conceptualizing fidelity of the Goal Setting Challenge App, a technology-delivered, intervention based on the Self-Determined Learning Model of Instruction. To develop the fidelity framework, we adopted elements of the standard setting process. Specifically, we created performance level descriptors that describe "good enough" fidelity for three fidelity dimensions: adherence, quality of delivery, and participant responsiveness. Using data from a pilot trial, we present information related to participant responsiveness. We discuss how this could direct future research to support effective adoption of technology-delivered interventions by teachers and schools to advance outcomes.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1475511
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGkzMPqQXtBtm01Lu6fBSjxAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDLlxzlelyDk3Hp4ikgIBEICBmsm3H4yMs5oMXaW-itFj81tzxrCgKtH97LMkPSRDj3W94rEXf5JgToNFJ4jh7yIVPt_-vCSMx1NEsx4Y6gDRBCnoRlpdxCNfOPyDIHxfHptTGmy-FIULahkpFEn7WPv5T5tWhPTUpy9wf8NZC3zyUEjOPwRt7b0OmYnpsAM3Z0_8GFfbRLS2od7o9c20hYH0sn0CogioTtpKYZs=
Text:
  Availability: 1
  Value: <anid>AN0185588280;[hcf6]01jun.25;2025Jun03.03:22;v2.2.500</anid> <title id="AN0185588280-1">Conceptualizing Fidelity of Implementation of a Technology-Delivered Self-Determination Intervention </title> <p>As technology becomes a more central support for students with disabilities in inclusive schools, there is a need to understand fidelity of implementation of technology-delivered interventions. We propose a framework for conceptualizing fidelity of the Goal Setting Challenge App, a technology-delivered, intervention based on the Self-Determined Learning Model of Instruction. To develop the fidelity framework, we adopted elements of the standard setting process. Specifically, we created performance level descriptors that describe "good enough" fidelity for three fidelity dimensions: adherence, quality of delivery, and participant responsiveness. Using data from a pilot trial, we present information related to participant responsiveness. We discuss how this could direct future research to support effective adoption of technology-delivered interventions by teachers and schools to advance outcomes.</p> <p>Keywords: self-determination; implementation fidelity; secondary education and transition; instructional technology; students with disabilities</p> <p>Measuring fidelity of implementation is essential not only to document that an intervention was implemented as intended but also to identify reasons why expected outcomes of research-based interventions are not obtained when scaling-up in practice ([<reflink idref="bib7" id="ref1">7</reflink>]). Despite a growing emphasis on reporting fidelity of implementation data in intervention research as well as calls to use these data to inform the enhancement of needed implementation supports ([<reflink idref="bib2" id="ref2">2</reflink>]; [<reflink idref="bib9" id="ref3">9</reflink>]; [<reflink idref="bib25" id="ref4">25</reflink>]; [<reflink idref="bib26" id="ref5">26</reflink>]), there is limited agreement on best practices to capture data on fidelity, report it in research studies, and analyze it to inform research and the implementation of evidence-based interventions to advance student outcomes in inclusive schools. This is particularly true of fidelity for complex interventions, which require flexibility and problem solving on the part of teacher implementers to integrate core components into their curriculum and local context while addressing the needs of individual students, particularly students with intellectual and developmental disabilities (IDD) planning for their transition from school to adult life ([<reflink idref="bib24" id="ref6">24</reflink>]). This is also true for technology-delivered interventions, which introduce new considerations in defining fidelity and its indicators as delivery shifts from teachers to technology, but teachers still play a role in supporting student access and engagement in inclusive schools ([<reflink idref="bib8" id="ref7">8</reflink>]; [<reflink idref="bib11" id="ref8">11</reflink>]).</p> <p>In self-determination research, the Self-Determined Learning Model of Instruction (SDLMI; [<reflink idref="bib23" id="ref9">23</reflink>]) has been identified as an evidence-based practice to enhance in-school and postschool outcomes for students with disabilities ([<reflink idref="bib10" id="ref10">10</reflink>]; [<reflink idref="bib13" id="ref11">13</reflink>]). The SDLMI is a complex intervention that requires trained implementers (e.g., general and special education teachers in school contexts) to problem solve how to implement its three core components (Student Questions, Teacher Objectives, and Educational Supports) in ways aligned with their curriculum and individualized student needs and supports. Reporting on fidelity of implementation of the SDLMI in the research literature has varied significantly over time, with different definitions of fidelity and measures adopted across studies ([<reflink idref="bib12" id="ref12">12</reflink>]); however, research has suggested that teachers of students with a range of disabilities, including IDD, can implement the SDLMI with fidelity with effective supports in inclusive general education classrooms as well as during transition planning ([<reflink idref="bib19" id="ref13">19</reflink>]). Recently, based on a review of the literature, [<reflink idref="bib24" id="ref14">24</reflink>] introduced a framework for conceptualizing fidelity of implementation of complex interventions like the SDLMI. The authors highlighted how the framework could guide planning for factors that could influence fidelity in inclusive schools and classrooms, advancing supports for teachers and schools to adopt evidence-based interventions to advance student outcomes. They also described how such a framework could enhance greater personalization of supports based on student needs and inform targeted training and professional development for teachers based on fidelity data if, for example, a greater need to focus on how to support students with IDD during self-determination instruction was identified. [<reflink idref="bib24" id="ref15">24</reflink>] recommended that three fidelity dimensions (i.e., adherence, quality of delivery, participant responsiveness) be assessed using diverse data sources to evaluate implementation of the SDLMI by teachers to inform their practice and support their sustained use of evidence-based interventions. General definitions of each of these dimensions from the fidelity literature, as well as definitions for the SDLMI specifically, are provided in the first three columns of Table 1.</p> <p>Table 1 Definitions of Fidelity Dimensions for Complex Interventions and Performance Level Descriptors for Fidelity of the Self-Determined Learning Model of Instruction (SDLMI) and the Goal Setting Challenge (GSC) App</p> <p>PHOTO (COLOR)</p> <p>As can be inferred from the definitions of fidelity dimensions for the SDLMI, teacher-delivered SDLMI instruction requires significant training and support for teacher implementers to ensure adherence and quality of delivery to the core components of the intervention and individualize to the support needs of students learning in inclusive schools. For example, students with IDD may have specific needs related to self-determination instruction, but unless this is infused into broader supports for fidelity of implementation, these needs may be missed or students denied access to instruction. Given that teachers consistently acknowledge the importance of self-determination instruction, but also state that they have limited time, training, and support for personalizing such instruction, there is an identified need to explore ways to promote access to evidence-based interventions like the SDLMI in innovative ways ([<reflink idref="bib14" id="ref16">14</reflink>]). Technology-delivered SDLMI instruction has the potential to change demands placed on teachers to learn and implement a complex intervention, and provide standardization of delivery of the core components aligned with student's needs ([<reflink idref="bib16" id="ref17">16</reflink>]).</p> <p>The Goal Setting Challenge (GSC) App was developed to address this need; its development is more fully described by [<reflink idref="bib14" id="ref18">14</reflink>]. The GSC App was specifically developed for the purpose of providing schools and teachers with a method for delivering goal setting and attainment instruction to students with disabilities in an innovative and time efficient way that could occur across settings in schools to advance transition planning. Initial research has shown the impacts of the App during transition instruction with students with a range of disability labels and support needs, including primarily students with learning disabilities but also students with IDD who are receiving transition instruction ([<reflink idref="bib20" id="ref19">20</reflink>]). To develop the GSC App, the process model of engagement, principles of instructional design, and culturally responsive practices (e.g., engaging end-users with disabilities from varying racial/ethnic backgrounds in development, including a range of "characters" in the App reflecting diverse identities, using examples of goals and action steps that reflected a range of family and student values) were used to "translate" the SDLMI and its core components into a web-based application that leverages cutting-edge instructional technologies and supports to deliver the SDLMI core components via technology. The GSC App includes 14 web-based lessons, including two introductory lessons and 12 lessons aligned with the 12 Student Questions organized into the three SDLMI phases (Phase 1—Set a Goal; Phase 2—Take Action; Phase 3—Adjust Goal or Plan; [<reflink idref="bib23" id="ref20">23</reflink>]). Accessibility and engagement features are embedded in the GSC App (e.g., multiple student response options, varying activity formats, predictability in structure) to address student needs in accessing the SDLMI core components (Student Questions, Teacher Objectives, Educational Supports). Development was guided not only by best practices in technology development ([<reflink idref="bib8" id="ref21">8</reflink>]) but also by sustained input from end-users (i.e., students with disabilities) and their teachers (see [<reflink idref="bib14" id="ref22">14</reflink>] for further information).</p> <p>Each GSC App lesson was designed to take students approximately 20 min to complete. To complete one GSC App challenge (i.e., setting and going after one goal by completing all 14 lessons) during one semester, students complete approximately two lessons a week. The GSC App was designed to be used repeatedly, across semesters. After the first semester of use, students can work to set new goals as well as modify goals or plans from previous semesters within the GSC App. Teachers can track students' progress in the GSC App using a Teacher Dashboard. The Teacher Dashboard uses metadata to provide information on where each student is in the GSC App. Teachers can also review students' responses to activities in each lesson to provide feedback. Preliminary research suggests that the GSC App shows promise for impacting student goal attainment outcomes, and teachers report finding the GSC App is time efficient and engaging for their students ([<reflink idref="bib15" id="ref23">15</reflink>]; [<reflink idref="bib20" id="ref24">20</reflink>]). However, there are additional questions that need to be addressed, particularly about measuring the fidelity of a complex intervention delivered via technology. Such work is needed to support teachers to embed this technology in their curriculum to support transition planning and to examine the association between implementation and student outcomes in-school and postschool ([<reflink idref="bib3" id="ref25">3</reflink>]).</p> <p>While on the surface, it may seem as though it could be easier to measure fidelity of implementation for web-based technology interventions given metadata collected, this may not always be the case ([<reflink idref="bib8" id="ref26">8</reflink>]). Factors related to teacher behaviors and their support for the App, as well as student engagement with the App, also need to be considered. In the fidelity of implementation literature for teacher-delivered interventions, adherence and quality of delivery are typically thought of as fidelity components that describe teacher behaviors, as it is teacher delivery that is evaluated. With the GSC App, these factors are built into the App and standardized through the web-based delivery format, but teachers still play a role in supporting students to access the App and complete lessons aligned with the class curriculum and implementation schedule, when used in school context. Teachers also support generalization of learning to other contexts (e.g., transition planning, academic learning; [<reflink idref="bib21" id="ref27">21</reflink>]) and need to be able to individualize supports, based on student support needs and instructional goals in inclusive settings. For example, teachers may need to support students with a range of disability labels and support needs during transition instruction. Planning for inclusive supports that can be personalized for student's specific needs is critical. Too often, the needs of students with IDD are not systematically considered in creating supports for technology-based or self-determination instruction. For these reasons, the roles teachers play in supporting students to engage with the App are relevant to fidelity ([<reflink idref="bib11" id="ref28">11</reflink>]).</p> <p>As technology becomes a more common way to deliver educational interventions, it also becomes critical to identify robust ways in which fidelity can be measured through indicators of user engagement collected by technology ([<reflink idref="bib4" id="ref29">4</reflink>]; [<reflink idref="bib11" id="ref30">11</reflink>]). Participant responsiveness as a fidelity dimension describes a student's engagement and involvement in the intervention (i.e., student engagement with the App). It goes beyond just the amount of content received by students, but how they engaged with the content. Although participant responsiveness within a teacher-delivered intervention is typically reflected in observations of student engagement and response to instruction, a technology-delivered intervention, such as the GSC App, collects metadata that allow teachers and researchers to access different indicators of student responsiveness ([<reflink idref="bib4" id="ref31">4</reflink>]; [<reflink idref="bib6" id="ref32">6</reflink>]). This allows for exploration of students' engagement and enactment of skills (e.g., student response time and written or verbal responses to activities within the App). The GSC App collects significant amounts of metadata that feed into the Teacher Dashboard as well as researcher "backend" data that can be downloaded to analyze information on use for research purposes ([<reflink idref="bib4" id="ref33">4</reflink>]; [<reflink idref="bib6" id="ref34">6</reflink>]). For example, information can be accessed on student engagement in the App, such as, time spent on each lesson, completion of activities used to deliver core components, and responses provided to activities (e.g., Student Questions). Teachers can then infuse this into the supports they provide students as they engage with the App and as they support generalization of learning to other areas of transition planning.</p> <p>However, limited literature exists on how to leverage these metadata to define fidelity indicators and support teachers. This is particularly nuanced for the GSC App, given the decisions made in designing the App to support students with a wide range of disabilities (e.g., learning disabilities, IDD) to use the App across contexts with a variety of goals. For example, to finish each lesson, students must answer questions confirming they have met the Lesson Objectives (one of the core SDLMI components), as well as provide a response to each Student Question (another core SDLMI component; Lessons 3–14). This does not vary because if students do not confirm they met the Lesson Objectives or do not provide a response to the Student Question, they are required to review the lesson again. However, the quality of this engagement can vary as the system cannot yet screen the quality of the responses provided (e.g., whether a student provides a meaningful response to the Student Question). Evaluating quality of these responses goes beyond simply aggregating the metadata and requires additional analysis and coding ([<reflink idref="bib18" id="ref35">18</reflink>]) as well as consideration of how teachers support individual and groups of students in their classes to engage and progress in the GSC App using the Dashboard.</p> <p>Given these complexities, we developed a framework for conceptualizing fidelity of implementation of a technology-delivered self-determination intervention, the GSC App, that we believe can also inform efforts to advance high fidelity implementation in inclusive settings of other technology-delivered interventions. To develop this framework, we began by exploring general definitions of fidelity for complex interventions and performance level descriptors (PLDs) of the SDLMI that had already been developed (see Table 1). In general, performance levels are categories into which implementers, who are being assessed, are classified based on their scores ([<reflink idref="bib27" id="ref36">27</reflink>]). These categories can be used to inform ongoing needs for supporting teachers or students to engage with the App and its content. Then, we developed preliminary PLDs for fidelity of the GSC App. Next, we used these PLDs to develop preliminary indicators of each fidelity component; we also identified potential data sources for the indicators (e.g., metadata, teacher surveys, observations; [<reflink idref="bib3" id="ref37">3</reflink>]; [<reflink idref="bib4" id="ref38">4</reflink>]). We provide descriptive data on identified indicators collected from GSC App implementation in a small, pilot trial ([<reflink idref="bib20" id="ref39">20</reflink>]). Implications for assessing and using fidelity data are described throughout, including ways this could drive the supports provided to teachers and students using the GSC App during transition planning.</p> <hd id="AN0185588280-2">Developing a Fidelity Framework for the GSC App</hd> <p>To develop a fidelity framework for the GSC App, we reviewed the available literature on fidelity and technology-delivered interventions, particularly how the use of metadata has been conceptualized for other interventions (e.g., [<reflink idref="bib11" id="ref40">11</reflink>]) as well as the literature that informed the development of the SDLMI Fidelity Framework ([<reflink idref="bib24" id="ref41">24</reflink>]). We found that (a) there were differences in how dimensions of fidelity are conceptualized and defined across teacher-delivered and technology-delivered interventions, and (b) the literature on fidelity of web-based applications to inform fidelity focused on metadata did not robustly consider other factors related to teacher behaviors that support implementation and student responses to the instruction delivered through technology ([<reflink idref="bib3" id="ref42">3</reflink>]; [<reflink idref="bib4" id="ref43">4</reflink>]).</p> <p>To develop the fidelity framework for the GSC App, we adopted elements of the standard setting process (SSP). Standard setting refers to the process of developing cut scores on tests ([<reflink idref="bib5" id="ref44">5</reflink>]). The function of cut scores is to split the distribution of test scores into categories, which may provide a basis for score interpretations and uses ([<reflink idref="bib1" id="ref45">1</reflink>]). The SSP has been extensively applied to educational and occupational tests ([<reflink idref="bib27" id="ref46">27</reflink>]) but recently has been gaining traction in establishing fidelity standards for educational interventions, including the SDLMI ([<reflink idref="bib22" id="ref47">22</reflink>]). Broadly, the SSP includes: (<reflink idref="bib1" id="ref48">1</reflink>) identifying the purpose of the fidelity indicators and measure being established; (<reflink idref="bib2" id="ref49">2</reflink>) choosing an appropriate method; (<reflink idref="bib3" id="ref50">3</reflink>) selecting a facilitator and a panel, and familiarizing the panel with the measure and the process; (<reflink idref="bib4" id="ref51">4</reflink>) developing PLDs (or, if already developed, training the panel on PLDs); (<reflink idref="bib5" id="ref52">5</reflink>) conducting rounds of the panel's work on determining cut scores; (<reflink idref="bib6" id="ref53">6</reflink>) collecting the panel's feedback as well as their level on confidence in the result; (<reflink idref="bib7" id="ref54">7</reflink>) creating technical documentation that allows one to evaluate the validity of the cut scores; and (<reflink idref="bib8" id="ref55">8</reflink>) making a recommendation to a policy maker, whose responsibility it is to make a final decision (adapted from [<reflink idref="bib17" id="ref56">17</reflink>]).</p> <p>In this article, we focus on the first four steps, particularly Step 4 (development of PLDs). A unique feature of our work is the absence of an already developed measure of fidelity. In contrast to typical standard setting studies where measures are available prior to the beginning of the SSP, we intend to use the PLDs to guide the development of indicators, given the newness of delivering self-determination instruction through technology via the GSC App and the critical need to identify ways to identify teacher and student support needs to advance adoption in inclusive schools during transition planning. It should also be noted that neither the PLDs nor indicators described in this article should be considered final and must be further refined as ongoing work and implementation of the GSC App occurs.</p> <hd id="AN0185588280-3">Applying the Standard Setting Process to the GSC App</hd> <p>In this section, we describe the implementation of the first four steps of the SSP to the development of the GSC App fidelity measure. We focus specifically on Step 4 (i.e., development of PLDs).</p> <hd id="AN0185588280-4">Step 1: Purpose of the Fidelity Indicators and Measure</hd> <p>In applying the SSP to developing the GSC App fidelity measure, we aimed to (<reflink idref="bib1" id="ref57">1</reflink>) develop preliminary PLDs for the future GSC App fidelity measure, (<reflink idref="bib2" id="ref58">2</reflink>) develop preliminary indicators or identify directions for indicator development, (<reflink idref="bib3" id="ref59">3</reflink>) identify potential data sources for the indicators, and (<reflink idref="bib4" id="ref60">4</reflink>) develop cut scores later. Aligned with the standards setting work on the SDLMI fidelity measure ([<reflink idref="bib22" id="ref61">22</reflink>]), we used two performance levels: sufficient (i.e., "good enough") and nonsufficient (i.e., not "good enough") levels of implementation of the GSC App. Again, the goal of these performance levels is to be able to inform the targeting of supports to advance implementation and inclusive supports. We conceptualized fidelity at the teacher level, as we assumed that teachers would be the primary driver of access to and feedback on quality of responses in the App during transition planning.</p> <hd id="AN0185588280-5">Step 2: Choice of Method</hd> <p>In general, SSP can be classified into two broad categories: content-based methods and methods based on external evidence (e.g., data about completion of performance based tasks that do not require judgment; [<reflink idref="bib5" id="ref62">5</reflink>]). Content-based methods rely mainly on judgments about test items or test-takers; external evidence (if used) is used at the end of the SSP to evaluate reasonableness of cut scores. In the context of fidelity measurement, it is important for PLDs to describe what "good enough" fidelity looks like; thus, we used content-based methods. However, in future research, we will also aim to explore relations to external evidence (e.g., student outcomes, teacher performance of specific tasks related to App delivery).</p> <hd id="AN0185588280-6">Step 3: Facilitator and Panel</hd> <p>The third author served as a facilitator of the panel meetings given methodological expertise. Our expert panel included six experts with extensive knowledge of the SDLMI, the GSC App and its development, and fidelity and implementation science. The meetings occurred over an 8-month period and finished when there was consensus on the PLDs.</p> <hd id="AN0185588280-7">Step 4: Development of PLDs</hd> <p>In defining PLDs for the GSC App, we began with the framework developed for the SDLMI, as it was informed by a review of the fidelity literature and has informed teacher implementation supports in inclusive general education settings and during transition planning. However, we recognized that changes would be needed to the framework based on how the GSC App delivers instruction. The PLDs developed by the expert panel for the GSC App are provided in the last column of Table 1. Table 1 includes notable distinctions in the PLDs for the fidelity dimensions for the GSC App compared to the SDLMI. These distinctions relate to the different features that are available in technology-delivered interventions that can enable standardization in the delivery of the core components alongside individualized supports based on students' responses in the GSC App. The GSC App includes elements of Universal Design for Learning and best practices in instructional technology that were integrated into delivery of the lessons to ensure adherence and quality of delivery. Specifically, students are required to go through every page of the GSC App as well as provide a response to each activity (including Student Questions) embedded in a lesson before they can move forward or complete the lesson. If students do not complete an activity, they are consistently returned to that page and not allowed to move forward until the activity is completed. This approach standardizes the delivery and ensures students receive the same instruction on the core components built into each lesson, although student responsiveness to activities in the App could vary the same way they can during teacher-delivered SDLMI instruction (described subsequently). It is important to note that there was flexibility built into the GSC App to support how students could engage in lessons and activities to make it accessible to students with a range of support needs; however, each flexible option was designed to ensure the same core components were delivered (e.g., lessons could be read aloud or read by students, students could write or verbally record responses).</p> <p>Thus, the panel determined that adherence and quality of GSC App content delivery were standardized and not applicable for fidelity measurement. However, the panel also determined that teachers' contributions to the delivery of the GSC App were considerable as they plan curriculum and provide access to the App in school contexts (e.g., setting up accounts, allocating instruction time, sharing the purpose of the App, providing feedback by reviewing data in the Teacher Dashboard, supporting generalization). Thus, specific GSC App teacher behaviors were determined to be part of the PLDs for adherence and quality of delivery. As highlighted in Table 1, teacher behaviors were divided into what teachers do (a) before students begin engaging with the GSC App (e.g., setting up schedule, introducing purpose of the GSC App to students); and (b) during GSC App implementation (e.g., monitoring students' progress and responses using the Teacher Dashboard, supporting high quality delivery and engagement).</p> <p>The third fidelity dimension, participant responsiveness (i.e., student responses and engagement with the App), was determined to be a central dimension by the panel. Student behaviors as they engage and respond to activities (including Student Questions) are critical to understanding their responsiveness. Unlike teacher-delivered interventions, like the SDLMI where student responsiveness was observed and rated during classroom instruction, the panel determined metadata could replace traditional observational data collection ([<reflink idref="bib3" id="ref63">3</reflink>]; [<reflink idref="bib4" id="ref64">4</reflink>]; [<reflink idref="bib11" id="ref65">11</reflink>]).</p> <hd id="AN0185588280-8">Development of Indicators of Fidelity and Identification of Data Sources</hd> <p>After agreeing on the initial PLDs, the expert panel began to define indicators for each of the fidelity dimensions and identified data sources for each indicator (see Table 2). These indicators, data sources, and PLDs were iteratively developed and refined by the same panel. To inform this process, we reviewed implementation information and data from the development ([<reflink idref="bib14" id="ref66">14</reflink>]), feasibility testing ([<reflink idref="bib15" id="ref67">15</reflink>]), and pilot trial ([<reflink idref="bib20" id="ref68">20</reflink>]) of the GSC App, including (a) training materials, (b) metadata collected in the App, (c) rubrics developed to analyze the quality of student responses to activities (including Student Questions), (d) surveys completed by teachers on how they supported implementation of the App in their classes, and (e) student perceptions of their engagement with the App. We also explored new and additional indicators or data sources that might be needed beyond existing data sources. Table 2 provides an overview of key indicators identified by the expert panel for each of the fidelity dimensions, as well as proposed sources for data collection to measure each indicator.</p> <p>Table 2 Goal Setting Challenge (GSC) App Fidelity Dimension Indicators and Data Sources</p> <p>PHOTO (COLOR)</p> <hd id="AN0185588280-9">Adherence</hd> <p>For the dimension of adherence, the indicators focused on GSC App teacher behaviors. First, part of the standardized GSC App training involves supporting teachers to develop an implementation schedule that details when students will engage in GSC App lessons and how time will be allocated throughout the semester for them to complete at least one GSC App challenge (i.e., all 14 lessons). The existence of these schedules and their implementation serves as indicators that reflect adherence to GSC App instruction.</p> <p>Further, teachers are provided, during training, a sample lesson plan to deliver "pre-instruction" (i.e., GSC App Overview Lesson) focused on defining the purpose of the GSC App, why self-determination and goal setting are relevant to their lives, and how the GSC App connects to other areas of their education (e.g., setting goals in inclusive content classes, individualized education program and transition planning). The Overview Lesson also (a) highlights how students set up accounts; (b) communicates the expected dosage of participation (i.e., how often the student will log into the GSC App each week, how long the student will engage with the GSC App each session); and (c) introduces the implementation schedule (i.e., anticipated timeline for completing one GSC App challenge during the semester). We also developed preliminary versions of standardized survey questions that can be used to gather teacher self-reported data on delivery of the GSC App Overview Lesson, as well as an observation protocol that external fidelity observers can use to document whether the lesson was delivered as intended.</p> <p>After pre-instruction, there are also key indicators of fidelity as teachers support students to engage with the GSC App. Specifically, teachers are expected to login and use the Teacher Dashboard to track students' progress, aligned with the implementation schedule. Metadata from the GSC App provides information on teachers' behaviors in the App and can be aggregated for fidelity reporting and analyses, although this was not collected during the initial pilot trial and will need to be evaluated in future work. During training, teachers received information on how to talk with students during other instructional times about their goals set in the GSC App, as well as how to provide additional Educational Supports if teachers observe that students are not meaningfully engaging in the GSC App based on their review of student responses in the Teacher Dashboard. Also, the GSC App User's Guide ([<reflink idref="bib21" id="ref69">21</reflink>]) presents examples and guidance on how to provide additional Educational Supports and linkages to other learning.</p> <hd id="AN0185588280-10">Quality of Delivery</hd> <p>Indicators and data sources for quality of delivery by teachers using the GSC App align with and build on the adherence indicators, shifting the focus from <emph>whether</emph> the elements were delivered to <emph>how well</emph> they were delivered. This is similar to the distinction between adherence and quality of delivery for the SDLMI. For this reason, indicators for this dimension more robustly explore the degree to which teachers individualize instruction and supports to student's support needs, consistent with training protocols. As shown in Table 2, quality of delivery indicators assesses the degree to which (a) pre-instruction was individualized to students' needs; and (b) individualized Educational Supports, supports for engagement, and supports for generalization were provided and aligned with student needs. Metadata from the App can be used to document if teachers routinely review student goals in the App; teachers can be surveyed and observed to examine the degree to which these goals are referenced in other instructional activities to promote generalization and engagement in other educational areas.</p> <hd id="AN0185588280-11">Participant Responsiveness</hd> <p>Participant responsiveness was defined by student behaviors as they engaged in the GSC App. Metadata provide a rich source of data to document student engagement in the App. First, GSC App metadata provide log-in information that can be used to determine if students ever accessed the GSC App and, if yes, if they completed at least one challenge (all lessons) in the App. Metadata also provides information on time spent in the App and students written or verbal responses to the Student Questions, as well as to other activities embedded in the lessons. We developed a standardized coding framework ([<reflink idref="bib18" id="ref70">18</reflink>]) that teachers or research team members can use to evaluate the alignment of responses with lesson objectives. This can be used for fidelity reporting as well as by teachers to inform ongoing instructional supports needed.</p> <hd id="AN0185588280-12">Descriptive Data From the Pilot Trial of the GSC App</hd> <p>Using student data from the pilot trial of the GSC App ([<reflink idref="bib20" id="ref71">20</reflink>]), we engaged in an initial evaluation of our proposed participant responsiveness PLDs and associated indicators. Unfortunately, the teacher and observational fidelity protocols aligned with the adherence and quality of delivery indicators were developed using information gathered during feasibility testing and the pilot trial, and the Teacher Dashboard was created in response to a need identified during feasibility testing and not yet functional in the pilot trial. Thus, these data sources for our fidelity framework were not yet available, could not be evaluated, and should be considered for future research. However, by evaluating the participant responsiveness indicators, we hope to inform ongoing refinement of these indicators and the framework as a whole to further inform the supports provided to teachers seeking to advance inclusive supports for self-determination as students plan for the transition from school to adult life.</p> <hd id="AN0185588280-13">School and Participant Context</hd> <p>The data were collected in a small, cluster (schools) randomized control trial of the GSC App (GSC App vs. business-as-usual transition instruction) in the 2020–2021 school year in one Midwest and two Southeast states in the United States. Eight schools were randomly assigned to the intervention (GSC App) condition (see [<reflink idref="bib20" id="ref72">20</reflink>] for more information about the sample); 111 students accessed the GSC App during the first semester of implementation. This sample was used to explore data on participant responsiveness Indicator 1b (i.e., completion of the first GSC App challenge within one academic semester; as described in the following sections. These 111 students were taught by 16 teachers; these teachers had, on average, 6.9 students (<emph>SD</emph> = 6.0), ranging from 1 to 26. Teachers were supporting students during transition planning activities, either during small group support time or transition class instructional time. Applications in other, inclusive settings need to be further researched. Out of the 111 students who logged in at any point during the semester, 59 (53.2%) students completed the first GSC App challenge (all 14 lessons) during the first semester of implementation. There was school, teacher, and student attrition during the pilot trial, largely because of the ongoing impacts of the COVID-19 pandemic on schools (e.g., teaching shortages, restructuring classes); teachers (e.g., moving classrooms, schools, leaving teaching); and students (e.g., low attendance, exiting school, needing additional supports for academic instruction during time when they accessed the GSC App). The 59 students were taught by 14 teachers; these teachers had, on average, 4.2 students (<emph>SD</emph> = 2.6), ranging from 1 to 8 students.</p> <p>Of the 59 students, 57.6% identified as male and 28.8% as female. The sample also identified as White/European American (40.7%), Black/African American (30.5%), American Indian or Alaskan Native (3.4%), Asian (1.7%), or two or more races (6.8%; the remaining data were missing or outside of these categories). Further, 17.0% of students were of Hispanic or Latino/Latina/Latinx origin, whereas 72.9% were not. Additionally, 40.7% of students self-reported not having a disability, 42.4% reported having a disability, and 17.0% did not provide this information. Students with disabilities had a range of disability labels, with the most common being learning disabilities (11.9%), autism (8.5%), and multiple disabilities (5.1%). However, all students were receiving the GSC App during targeted instruction for transition planning, consistent with receiving special education services. As this was how the GSC App was designed to be delivered, our focus was on exploring the impacts of the App for students receiving transition instruction which would include a range of students, inclusive of students with IDD, who are in need of high quality self-determination instruction.</p> <hd id="AN0185588280-14">Data Sources and Operationalization of Participant Responsiveness Fidelity Indicators</hd> <p>As shown in Table 2, we created five indicators of participant responsiveness which are grouped into two categories aligned with data sources.</p> <hd id="AN0185588280-15">Metadata</hd> <p>As described, metadata are available in the GSC App and can be used to track student engagement and progress. Our first Indicator (1a) is designed to measure the percentage of assigned students who accessed the GSC App (out of all assigned students taught by the teacher throughout a given semester). Given COVID-19 impacts, we had an original class roster in the pilot study, but we did not have reliable information about the reason for why a given student never accessed the GSC App (e.g., did teacher determine it was not appropriate for the student, did student(s) move to another class). We encouraged ongoing data collection, and we focused on Indicators 1b (i.e., complete the first GSC App challenge [all 14 GSC App lessons] within one academic semester) and 1c (i.e., amount of time that students spent on the first GSC App challenge within one academic semester) for the purposes of this article. Indicator 1b was operationalized as the percentage of students (of a given teacher) who completed all activities in one GSC App challenge within one semester (out of all students who accessed the GSC App). Indicator 1c was operationalized as the average amount of time that students (of a given teacher), who completed the first challenge, spent in the GSC App on the first challenge. Both indicators were aggregated at the teacher level by averaging student-level data for each teacher.</p> <hd id="AN0185588280-16">Quality of Student Responses to GSC App Activities</hd> <p>Our second category of Indicators (2a and 2b) evaluated the quality of student responses to activities in the GSC App. The App has a range of activities where students provide written responses to open-ended questions. These activities also included the Student Questions, one of which was presented at the end of Lessons 3 through 14. Therefore, we developed a standardized coding process ([<reflink idref="bib18" id="ref73">18</reflink>]) to evaluate the quality of (a) student responses to activities other than the Student Questions that are embedded in GSC App lessons to meet lesson objectives (Indicator 2a); and (b) student responses to the Student Questions presented at the end of each lesson (Indicator 2b). Quality coding for Indicator 2a focused on whether responses to activities were not, partially, or fully aligned with the objective of each activity (e.g., identify two strengths, identify action steps toward a goal). For Indicator 2b, Student Questions, responses were rated based on if (<emph>yes/no</emph>) student's response related to their goal area and demonstrated a step toward solving the problem posed in the question. A total of 54 rating questions were developed for 38 activities (including Student Questions). However, four questions for the corresponding four activities (non-Student Questions) were excluded from the indicator calculations because the questions did not allow for variability in their ratings given their design in the GSC App. The procedure for calculating these indicators included multiple steps: (<reflink idref="bib1" id="ref74">1</reflink>) putting all ratings on the same scale (0–1) by dividing each rating by its maximum possible value; (<reflink idref="bib2" id="ref75">2</reflink>) calculating activity-level ratings by averaging the ratings for the questions developed for each activity (all activities were conceptualized as having the same weight); (<reflink idref="bib3" id="ref76">3</reflink>) separating activities into two groups: activities that are not Student Questions versus. Student Questions; (<reflink idref="bib4" id="ref77">4</reflink>) for each group of activities, calculating student-level ratings by averaging activity-level ratings for each student; and (<reflink idref="bib5" id="ref78">5</reflink>) also for each group of activities, calculating teacher-level ratings by averaging student-level ratings for each teacher.</p> <hd id="AN0185588280-17">Results</hd> <p>For Indicator 1b (i.e., complete the first GSC App challenge within one academic semester), we found that, on average, 65.3% of students (of a given teacher), who accessed the GSC App, completed the first GSC App challenge during an academic semester (<emph>SD</emph> = 40.3). Two teachers (12.5%) did not have any students who completed the first challenge, and for seven teachers (43.8%), all students completed the first challenge. Next, for Indicator 1c (i.e., amount of time that students spent on the first GSC App challenge within one academic semester), we found that students (of a given teacher), who completed the first challenge during the academic semester, spent on average 187.2 min (<emph>SD</emph> = 66.1), ranging from 98.3 to 359.8 min.</p> <p>For Indicator 2a, we found that the quality of responses to activities across lessons, other than Student Questions, students (of a given teacher), who completed the first challenge during the academic semester, was rated on average as 0.9 (<emph>SD</emph> = 0.1), ranging from 0.6 to 1.0. For Indicator 2b, the quality of responses to Student Questions, students (of a given teacher), who completed the first challenge during the academic semester, was rated on average as 0.8 (<emph>SD</emph> = 0.1), ranging from 0.6 to 1.0.</p> <hd id="AN0185588280-18">Discussion</hd> <p>The GSC App is unique as it is a technology-delivered self-determination intervention. Few technology-based interventions have been developed to support self-determination and transition planning that can benefit all students with disabilities but can also be personalized to the unique support needs of each student, including students with IDD. Technological supports have the potential to advance the personalized supports available to students to set and go after goals for inclusive, postschool lives. They also have the potential to promote access for all students, including students who may not always be included in technology development, like students with IDD. However, ways to support the integration of technology-based supports into the curriculum, including ways to document and address problems with fidelity of implementation, are critically needed. We hope this work can have relevance to work that promotes equitable access to high quality technology, self-determination, and transition instruction for students with IDD. In this article, we described the development and preliminary data on a framework for measuring fidelity implementation of the GSC App that could be used to guide ongoing supports to teachers to enhance student responsiveness and outcomes while engaging with the GSC App during transition planning. But there is a need for ongoing work to fully evaluate this framework as well as for large scale investigations of the relationship between GSC App fidelity and student outcomes, including exploration of ways to overcome barriers to access for students with IDD and exploration of factors that predict fidelity of implementation for this group.</p> <hd id="AN0185588280-19">Limitations and Implications for Future Research</hd> <p>In creating our PLDs and indicators, we did not rely solely on App-provided student data (i.e., metadata) but also attempted to capture the role that teachers play in promoting student responsiveness as there is an increasing need for effective supports for teachers, particularly teachers that are supporting students with IDD to advance transition planning in inclusive schools. Specifically, we developed fidelity indicators to measure teacher roles in adherence (e.g., creating an implementation schedule, communicating the schedule to students, monitoring student progress in the app, checking in with students about their progress) as well as in quality of delivery (e.g., providing a clear justification for the schedule, having conversations with students about their goals, facilitating generalization to other content areas). For the purposes of this article, we solely focused our fidelity data collection on student responsiveness given the limitations of the GSC App Teacher Dashboard at the time of this work. However, given the amount of time and cost demands of collecting fidelity data, there is often an appeal to simply focus on student responsiveness and conceptualizing fidelity at the student level. However, assessing only student responsiveness cannot provide information on what supports are effectively advancing student engagement and learning, even in a technology-delivered intervention in schools. Conceptualizing fidelity at the student level does not provide information about a teacher and their needed supports, as teachers are the ones who are supporting the implementation of the intervention and may also need supports to advance their practice. Future work is needed to determine if all adherence and quality of delivery indicators are equally relevant for technology-delivered interventions and fully aligned with student responsiveness indicators, student support needs, and ultimately student outcomes.</p> <p>Measuring student responsiveness for technology-delivered interventions may seem straightforward as there are rich metadata provided by the GSC App. However, we found that indicators directly from metadata are not sufficient to fully represent the construct of student responsiveness. First, we found that metadata as a data source is not sufficient because it does not provide all data necessary to measure student access to the App (Participant Responsiveness Indicator 1a). Specifically, in addition to the information about who accessed the App (available in metadata), we also need information about who was enrolled in the intervention and whether they were continuously attending the class during the semester. Second, raw metadata is also insufficient as the App itself cannot code and evaluate the quality of student responses to activities (including Student Questions)—although emerging artificial intelligence technologies may change this. For now, the coding of metadata will likely be needed by researchers or teachers to measure quality indicators of student responsiveness. Future work is needed to further explore whether all relevant indicators of responsiveness have been identified and whether the procedures for calculating them are appropriate and time effective. Work should also explore if indicators vary across student groups, including students with IDD.</p> <p>The available data for the participant responsiveness indicators does provide information on how students engaged in the App and suggests directions for future research, even if there are limitations in these data. For example, for most teachers (62.5%), approximately half of their students completed the first GSC App challenge during the semester—suggesting attrition may be influenced by teacher-level factors warranting future research. This is promising, particularly given the COVID-19 impacts on this study. However, we also found variability among teachers in terms of the average time their students spent on the first GSC App challenge and the quality of their students' responses, although, notably, the quality rates were above the scale midpoint for all teachers. Overall, when students logged in and completed the GSC App, they seemed to show meaningful levels of engagement and high quality of responses, although there was variability. This suggests there may be more ways to engage teachers in supporting students as they engage in the GSC App as well as determining if there are features of the App that should be further considered to promote responsiveness, given the numbers of students that did not complete their first challenge. We also need to further evaluate the impact of student level factors on engagement and responsiveness, particularly how to align supports with student's disability-related support needs and the demands of inclusive contexts as the sample size was too small to present these comparisons.</p> <p>Once the indicators are finalized, we aim to move to the next steps of the SSP and focus on developing cut scores to enable us to tailor supports to teachers using the GSC App. Specifically, using content-based methods, we aim to determine what level of implementation is "good enough." Once the cut scores are set, several research questions of interest could be explored. Of particular importance is an exploration of the relations to student outcomes. Specifically, we aim to investigate if there is a difference in student outcomes between teachers who implemented the GSC App intervention with fidelity compared to those who did not implement the intervention with fidelity as well as factors that might influence this such as student support needs or disability labels. This information will help us understand which dimensions of fidelity might be the most important for student outcomes and, hence, what factors should receive more attention during teacher training. We can also further explore various teacher and student factors that influence responsiveness. Another important question is related to how fidelity can best be supported. Specifically, there is a need to explore how to support effective teacher training, what additional teacher supports are needed, and whether the intervention complexity needs to be adjusted based on student characteristics or support needs. Overall, there are a number of research questions that need to be further explored, and this study provides a starting point for doing so and continues to move research forward in the most efficacious ways to support teachers and students to engage in self-determination interventions, specifically technology-delivered self-determination interventions.</p> <hd id="AN0185588280-20">Implications for Practice</hd> <p>More work is needed to understand, even for technology-delivered interventions, the role of teachers or other school staff in supporting young people with disabilities, particularly those with IDD, to engage with a technology-based intervention to ensure quality of responses and generalize what is being learned to other areas of their lives to support transition to adulthood. Ongoing exploration of the most time and cost-effective means to support students to meaningfully engage with the GSC App and document fidelity is essential to inform teacher training and supports. Exploring the linkages between adherence, quality of delivery, and participant responsiveness with student outcomes will be an important part of these explorations, to inform both research and practice and can advance equitable outcomes. Understanding what dimensions best predict outcomes will be important and will allow us to begin to ask questions and involve teachers in the process of learning what supports students and teachers need to integrate technology-driven instruction into learning and what supports have the most positive impacts for students. For example, do teachers need coaching and additional classroom supports for generalizing learning from the GSC App and does this lead to more positive student outcomes, and do these needs vary based on student support needs or other contextual factors? Do teachers still need to implement other interventions, such as the SDLMI, in other contexts, to support students to generalize their learning and do some students benefit more from more intensive instruction? Also, if students are not demonstrating high levels of responsiveness, what supports can teachers provide or what supports need to be built into the GSC App to advance student success? What student characteristics shape responsiveness? Understanding the intricacies and nuances of teacher- and technology-delivered interventions and the importance of fidelity of implementation can and will impact implementation moving forward with the ultimate goal of advancing the use of evidence-based interventions to support students to set and go after goals for their inclusive, self-determined futures.</p> <p> <emph>The research reported here was supported by the Institute of Education Sciences</emph>, <emph>U.S</emph>. <emph>Department of Education, through Grant R324A180012 to The University of Kansas. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education</emph>.<emph>The authors would like to thank all the students, teachers, schools, and families who participated in and informed this work. The authors would like to also thank the entire Kansas University Center on Developmental Disabilities team and partners across the country who supported this work and made this article possible.</emph></p> <ref id="AN0185588280-21"> <title> References </title> <blist> <bibl id="bib1" idref="ref45" type="bt">1</bibl> <bibtext> American Educational Research Association, American Psychological Association, & National Council for Measurement in Education.(2014). Standards for education and psychological testing.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref2" type="bt">2</bibl> <bibtext> Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., & Czajkowski, S. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23(5), 443–451. https://doi.org/10.1037/0278-6133.23.5.443</bibtext> </blist> <blist> <bibl id="bib3" idref="ref25" type="bt">3</bibl> <bibtext> Bonar, J. R. M., Wright, S., Yadrich, D. M., Werkowitch, M., Ridder, L., Spaulding, R., & Smith, C. E. (2022). Maintaining intervention fidelity when using technology delivery across studies. Computers, Informatics, Nursing, 38(8), 393–401. https://doi.org/10.1097/CIN.0000000000000625</bibtext> </blist> <blist> <bibl id="bib4" idref="ref29" type="bt">4</bibl> <bibtext> Breitenstein, S. M., Brager, J., Ocampo, E. V., & Fogg, L. (2017). Engagement and adherence with ezPARENT, an mHealth parent-training program promoting child well-being. Child Maltreatment, 22(4), 295–304. https://doi.org/10.1177/1077559517725402</bibtext> </blist> <blist> <bibl id="bib5" idref="ref44" type="bt">5</bibl> <bibtext> Cizek, G. J., & Earnest, D. S. (2015). Setting performance standards on tests. In Handbook of test development (2nd ed; pp. 228–254). Routledge.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref32" type="bt">6</bibl> <bibtext> Donkin, L., Hickie, I. B., Christensen, H., Naismith, S. L., Neal, B., Cockayne, N. L., & Glozier, N. (2013). Rethinking the dose-response relationship between usage and outcome in an online intervention for depression: Randomized controlled trial. Journal of Medical Internet Research, 15(10), e231. https://doi.org/10.2196/jmir.2771</bibtext> </blist> <blist> <bibl id="bib7" idref="ref1" type="bt">7</bibl> <bibtext> Fixsen, D. L., Blase, K. A., Duda, M. A., Naoom, S. F., & Van Dyke, M. (2010). Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. In Weisz J. R. & Kazdin A. E. (Eds.), Evidence-based psychotherapies for children and adolescents (2nd ed., pp. 435–450). Guilford Press.</bibtext> </blist> <blist> <bibl id="bib8" idref="ref7" type="bt">8</bibl> <bibtext> Gersten, R., & Edyburn, D. (2007). Defining quality indicators for special education technology research. Journal of Special Education Technology, 22(3), 3–18. https://doi.org/10.1177/016264340702200302</bibtext> </blist> <blist> <bibl id="bib9" idref="ref3" type="bt">9</bibl> <bibtext> Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71(2), 149–164. https://doi.org/10.1177/001440290507100202</bibtext> </blist> <blist> <bibtext> Hagiwara, M., Shogren, K., & Leko, M. (2017). Reviewing research on the Self-Determined Learning Model of Instruction: Mapping the terrain and charting a course to promote adoption and use. Advances in Neurodevelopmental Disorders, 1, 3–13. https://doi.org/10.1007/s41252-017-0007-7</bibtext> </blist> <blist> <bibtext> Helsabeck, N. P., Justice, L. M., & Logan, J. A. R. (2022). Assessing fidelity of implementation to a technology-mediated early intervention using process data. Journal of Computer Assisted Learning, 38(2), 409–421. https://doi.org/10.1111/jcal.12621</bibtext> </blist> <blist> <bibtext> Kiblen, J. C., Shogren, K. A., Zimmerman, K. N., Raley, S. K., Mansouri, M. C., & Alsaaed, A. H. (2023). A review of teacher fidelity measurement of the Self-Determined Learning Model of Instruction. Education and Training in Autism and Developmental Disabilities, 58(4), 422–440.</bibtext> </blist> <blist> <bibtext> Mazzotti, V. L., Rowe, D. A., Kwiatek, S., Voggt, A., Chang, W.-H., Fowler, C. H., Poppen, M., Sinclair, J., & Test, D. W. (2021). Secondary transition predictors of postschool success: An update to the research base. Career Development and Transition for Exceptional Individuals, 44(1), 47–64. https://doi.org/10.1177/2165143420959793</bibtext> </blist> <blist> <bibtext> Mazzotti, V. L., Shogren, K. A., Stewart-Ginsburg, J. H., Wysenski, D., Burke, K. M., & Hildebrandt, L. (2022). Development of the Goal Setting Challenge App: Engaging users to promote self-determination. International Journal of Disability, Development, and Education, 69(1), 331–351. https://doi.org/10.1080/1034912X.2021.1959022</bibtext> </blist> <blist> <bibtext> Mazzotti, V. L., Shogren, K. A., Stewart-Ginsburg, J. H., Wysenski, D., Hagiwara, M., Kwiatek, S., & Chapman, R. A. (2024). The Goal Setting Challenge App: Promoting self-determination through technology. Remedial and Special Education, 45(1), 3–17. https://doi.org/10.1177/07419325221147698</bibtext> </blist> <blist> <bibtext> Office of Education Technology.(2017). Reimaging the role of technology in education: 2017 National Education Technology Plan update. U.S. Department of Education.</bibtext> </blist> <blist> <bibtext> Pitoniak, M. J., & Morgan, D. L. (2017). Setting and validating cut scores for tests. In Secolsky C. & Denison D. B. (Eds.), Handbook on measurement, assessment, and evaluation in higher education (2nd ed., pp. 235–258). Routledge. https://doi.org/10.4324/9781315709307-21</bibtext> </blist> <blist> <bibtext> Raley, S. K., Townsend, R. J., Shogren, K. A., Mazzotti, V. L., Fredrick, D., Kwiatek, S. M., & Chapman, R. A. (2024). Examining student responses in technology-delivered self-determination intervention using the Goal-Setting Challenge App. The Journal of Special Education 58(1), 23–34. https://doi.org/10.1177/00224669231179241</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Burke, K. M., Anderson, M. H., Antosh, A. A., LaPlante, T., & Hicks, T. A. (2020). Examining the relationship between teacher perceptions of implementation of the SDLMI and student self-determination outcomes. Career Development and Transition for Exceptional Individuals, 43(1), 53–63. https://doi.org/10.1177/2165143419887855</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Mazzotti, V. L., Hicks, T. A., Raley, S. K., Gerasimova, D., Pace, J. R., Kwiatek, S. M., Fredrick, D., Stewart-Ginsburg, J. H., Chapman, R. A., & Wysenski, D. (2024). The Goal Setting Challenge App: Impact on transition goal attainment outcomes of students with disabilities. Career Development and Transition for Exceptional Individuals, 47(1), 4–18. https://doi.org/10.1177/21651434221146470</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Mazzotti, V. L., Kwiatek, S., Fredrick, D., Hildebrandt, L., Hagiwara, M., Wysenski, D., Stewart-Ginsburg, J. H., & Raley, S. K. (2022). The Goal Setting Challenge App user's guide: Supporting student delf-determination (Version 2.0). Kansas University Center on Developmental Disabilities.</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Pace, J. R., Hicks, T. A., Raley, S. K., & Lane, K. L. (2024). Setting standards for SDLMI fidelity: Promoting data-driven decision making to advance self-determination instruction. Psychology in the Schools, 61(2), 532–552. https://doi.org/10.1002/pits.23068</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Raley, S. K., Burke, K. M., & Wehmeyer, M. L. (2018). The Self-Determined Learning Model of Instruction: Teacher's guide. Kansas University Center on Developmental Disabilities.</bibtext> </blist> <blist> <bibtext> Shogren, K. A., Raley, S. K., Rifenbark, G. G., Lane, K. L., Bojanek, E. K., Karpur, A., & Quirk, C. (2021). The Self-Determined Learning Model of Instruction: Promoting implementation fidelity. Inclusion, 9(1), 46–62. https://doi.org/10.1352/2326-6988-9.1.46</bibtext> </blist> <blist> <bibtext> Toste, J. R., Logan, J. A. R., Shogren, K. A., & Boyd, B. A. (2023). The next generation of quality indicators for group design research in special education. Exceptional Children, 89(4), 359–378. https://doi.org/10.1177/00144029221150801</bibtext> </blist> <blist> <bibtext> What Works Clearinghouse. (2017). Standards handbook (Version 4.0). Institute of Education Sciences.</bibtext> </blist> <blist> <bibtext> Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cutscores: A manual for setting standards of performance on educational and occupational tests. Educational Testing Services.</bibtext> </blist> </ref> <aug> <p>By Karrie A. Shogren; Valerie L. Mazzotti; Daria Gerasimova; Darcy Fredrick; Stephen M. Kwiatek; Sheida K. Raley; Helena Rodrigues Ferreira; Aaron M. Beuoy and Tyler A. Hicks</p> <p>Reported by Author; Author; Author; Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib25" firstref="ref4"></nolink> <nolink nlid="nl2" bibid="bib26" firstref="ref5"></nolink> <nolink nlid="nl3" bibid="bib24" firstref="ref6"></nolink> <nolink nlid="nl4" bibid="bib11" firstref="ref8"></nolink> <nolink nlid="nl5" bibid="bib23" firstref="ref9"></nolink> <nolink nlid="nl6" bibid="bib10" firstref="ref10"></nolink> <nolink nlid="nl7" bibid="bib13" firstref="ref11"></nolink> <nolink nlid="nl8" bibid="bib12" firstref="ref12"></nolink> <nolink nlid="nl9" bibid="bib19" firstref="ref13"></nolink> <nolink nlid="nl10" bibid="bib14" firstref="ref16"></nolink> <nolink nlid="nl11" bibid="bib16" firstref="ref17"></nolink> <nolink nlid="nl12" bibid="bib20" firstref="ref19"></nolink> <nolink nlid="nl13" bibid="bib15" firstref="ref23"></nolink> <nolink nlid="nl14" bibid="bib21" firstref="ref27"></nolink> <nolink nlid="nl15" bibid="bib18" firstref="ref35"></nolink> <nolink nlid="nl16" bibid="bib27" firstref="ref36"></nolink> <nolink nlid="nl17" bibid="bib22" firstref="ref47"></nolink> <nolink nlid="nl18" bibid="bib17" firstref="ref56"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1475511
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Conceptualizing Fidelity of Implementation of a Technology-Delivered Self-Determination Intervention
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Karrie+A%2E+Shogren%22">Karrie A. Shogren</searchLink><br /><searchLink fieldCode="AR" term="%22Valerie+L%2E+Mazzotti%22">Valerie L. Mazzotti</searchLink><br /><searchLink fieldCode="AR" term="%22Daria+Gerasimova%22">Daria Gerasimova</searchLink><br /><searchLink fieldCode="AR" term="%22Darcy+Fredrick%22">Darcy Fredrick</searchLink><br /><searchLink fieldCode="AR" term="%22Stephen+M%2E+Kwiatek%22">Stephen M. Kwiatek</searchLink><br /><searchLink fieldCode="AR" term="%22Sheida+K%2E+Raley%22">Sheida K. Raley</searchLink><br /><searchLink fieldCode="AR" term="%22Helena+Rodrigues+Ferreira%22">Helena Rodrigues Ferreira</searchLink><br /><searchLink fieldCode="AR" term="%22Aaron+M%2E+Beuoy%22">Aaron M. Beuoy</searchLink><br /><searchLink fieldCode="AR" term="%22Tyler+A%2E+Hicks%22">Tyler A. Hicks</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Inclusion%22"><i>Inclusion</i></searchLink>. 2025 13(2):116-132.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: American Association on Intellectual and Developmental Disabilities. P.O. Box 1897, Lawrence, KS 66044-1897. Tel: 785-843-1235; Fax: 785-843-1274; e-mail: AJMR@allenpress.com; Web site: https://meridian.allenpress.com/inclusion
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 17
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Students+with+Disabilities%22">Students with Disabilities</searchLink><br /><searchLink fieldCode="DE" term="%22Inclusion%22">Inclusion</searchLink><br /><searchLink fieldCode="DE" term="%22Fidelity%22">Fidelity</searchLink><br /><searchLink fieldCode="DE" term="%22Goal+Orientation%22">Goal Orientation</searchLink><br /><searchLink fieldCode="DE" term="%22Self+Determination%22">Self Determination</searchLink><br /><searchLink fieldCode="DE" term="%22Standard+Setting%22">Standard Setting</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1352/2326-6988-13.2.116
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2326-6988
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: As technology becomes a more central support for students with disabilities in inclusive schools, there is a need to understand fidelity of implementation of technology-delivered interventions. We propose a framework for conceptualizing fidelity of the Goal Setting Challenge App, a technology-delivered, intervention based on the Self-Determined Learning Model of Instruction. To develop the fidelity framework, we adopted elements of the standard setting process. Specifically, we created performance level descriptors that describe "good enough" fidelity for three fidelity dimensions: adherence, quality of delivery, and participant responsiveness. Using data from a pilot trial, we present information related to participant responsiveness. We discuss how this could direct future research to support effective adoption of technology-delivered interventions by teachers and schools to advance outcomes.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1475511
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1475511
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1352/2326-6988-13.2.116
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 17
        StartPage: 116
    Subjects:
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Students with Disabilities
        Type: general
      – SubjectFull: Inclusion
        Type: general
      – SubjectFull: Fidelity
        Type: general
      – SubjectFull: Goal Orientation
        Type: general
      – SubjectFull: Self Determination
        Type: general
      – SubjectFull: Standard Setting
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
    Titles:
      – TitleFull: Conceptualizing Fidelity of Implementation of a Technology-Delivered Self-Determination Intervention
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Karrie A. Shogren
      – PersonEntity:
          Name:
            NameFull: Valerie L. Mazzotti
      – PersonEntity:
          Name:
            NameFull: Daria Gerasimova
      – PersonEntity:
          Name:
            NameFull: Darcy Fredrick
      – PersonEntity:
          Name:
            NameFull: Stephen M. Kwiatek
      – PersonEntity:
          Name:
            NameFull: Sheida K. Raley
      – PersonEntity:
          Name:
            NameFull: Helena Rodrigues Ferreira
      – PersonEntity:
          Name:
            NameFull: Aaron M. Beuoy
      – PersonEntity:
          Name:
            NameFull: Tyler A. Hicks
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 2326-6988
          Numbering:
            – Type: volume
              Value: 13
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Inclusion
              Type: main
ResultId 1