View in EDS HTML Full Text PDF Full Text

Evaluation of a Social-Emotional Learning Program in Conjunction with the Exploratory Application of Performance Feedback Incorporating Motivational Interviewing Techniques

Saved in:
Bibliographic Details
Title: Evaluation of a Social-Emotional Learning Program in Conjunction with the Exploratory Application of Performance Feedback Incorporating Motivational Interviewing Techniques
Language: English
Authors: Gueldner, Barbara, Merrell, Kenneth
Source: Journal of Educational & Psychological Consultation. 2011 21(1):1-27.
Availability: Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Peer Reviewed: Y
Physical Description: PDF
Page Count: 27
Publication Date: 2011
Document Type: Journal Articles
Reports - Evaluative
Descriptors: Feedback (Response), Middle School Students, Early Intervention, Interviews, Program Effectiveness, Prevention, Behavior Problems, Child Development, Program Descriptions, Resilience (Psychology), Symptoms (Individual Disorders), Motivation Techniques, Social Development, Emotional Development
DOI: 10.1080/10474412.2010.522876
ISSN: 1047-4412
Abstract: The prevention of and early intervention with internalizing problems are vital to children's overall health and development. Schools are strongly considered a venue where these services can be provided. The Strong Kids program was developed to increase youths' resiliency. This study examined the efficacy of Strong Kids with middle school students in terms of knowledge of healthy social-emotional behaviors, reduction in internalizing symptoms, and the exploratory use of performance feedback with features of motivational interviewing. Students who participated in the program significantly increased their knowledge. Negative symptoms did not change. Teachers implemented the program with strong fidelity. It is unclear at this time the extent to which it is necessary to use performance feedback with features of motivational interviewing to improve the implementation of highly structured social-emotional learning programs. (Contains 6 tables.)
Abstractor: As Provided
Number of References: 61
Entry Date: 2011
Accession Number: EJ917352
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwEwQ279_ZTggvehTFmMQ3CfAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDL5t6uLMn7DL3BSY2AIBEICBmvMfDtdR7vljYMmcmBbdNDhSKM6FAfcyB0D5y4M-S-CxmO5StQjiryUHwzOuoyRmXgc9JhB6wTpMP66rpTlxSMeXVSDkpQABybT4aTlciIsYOqt_G2ed-T_mANqsKGygU-fqqFfHzOQ-FyFt3etzDlBs7ds24jwc_GSEDKvKOxShZ8QDc5lAWf6PAqloTyMcusMqBWe5_F3Su00=
Text:
  Availability: 1
  Value: <anid>AN0059165621;7ma01jan.11;2019Feb13.13:07;v2.2.500</anid> <title id="AN0059165621-1">Evaluation of a Social-Emotional Learning Program in Conjunction With the Exploratory Application of Performance Feedback Incorporating Motivational Interviewing Techniques. </title> <sbt id="AN0059165621-2">Introduction</sbt> <p>The prevention of and early intervention with internalizing problems are vital to children's overall health and development. Schools are strongly considered a venue where these services can be provided. The Strong Kids program was developed to increase youths' resiliency. This study examined the efficacy of Strong Kids with middle school students in terms of knowledge of healthy social-emotional behaviors, reduction in internalizing symptoms, and the exploratory use of performance feedback with features of motivational interviewing. Students who participated in the program significantly increased their knowledge. Negative symptoms did not change. Teachers implemented the program with strong fidelity. It is unclear at this time the extent to which it is necessary to use performance feedback with features of motivational interviewing to improve the implementation of highly structured social-emotional learning programs.</p> <p>The mental health needs of school-age youth continue to be a significant area of concern when considering the transaction between typical developmental stressors and the ability to cope effectively. Middle school-age students, in particular, have unique challenges as they experience physical changes, cognitive maturation, and social pressures. Literature on school bullying ([<reflink idref="bib16" id="ref1">16</reflink>]) and identification of behavioral and emotional disturbances ([<reflink idref="bib27" id="ref2">27</reflink>]) suggests that internalizing problems, such as anxiety and depression, tend to emerge with more frequency in this age range. It is estimated that at any point in time, 4–6% of children may be experiencing an internalizing disorder ([<reflink idref="bib33" id="ref3">33</reflink>]), and over the course of childhood and adolescence, one in five girls and one in three boys will have experienced at least one episode of major depression ([<reflink idref="bib52" id="ref4">52</reflink>]). The World Health Organization (2004) estimated that by 2020, depression will account for the largest proportion among all neuropsychiatric conditions that lead to disability and lost years of life. Certainly, it is critical to provide effective treatments when individuals are experiencing problem symptoms with the goal of alleviating undeniable suffering. Looking beyond immediate needs, Coie, Miller-Jackson, and Bagwell (2000) emphasized the importance of changing this trajectory and focusing efforts on <emph>prevention</emph>.</p> <p>Efforts to enhance social and emotional competence via prevention efforts have emerged via a movement called <emph>social and emotional learning</emph> (SEL; Collaborative for Academic, Social, and Emotional Learning [CASEL], 2003). Not only do most educators agree that social and emotional skills are important but also strategies and programs that teach these skills have been found to positively influence learning and academic achievement ([<reflink idref="bib58" id="ref5">58</reflink>]). Additionally, attributes such as feeling attached to school, engagement, attendance, and overall behaviors that lead to positive school performance have been associated with acquiring social and emotional competence ([<reflink idref="bib61" id="ref6">61</reflink>]). Regarding internalizing problems, students who are persistently worried about test performance or are too sad and lethargic to get to school face serious obstacles to achieving positive outcomes. Specific programs, such as the Strong Kids curricula series, have been developed with these issues in mind: boosting students' abilities to cope with life stressors and enhancing social, emotional, and academic competence.</p> <p>The Strong Kids program is a 12-lesson, skill-based SEL program that was developed with the idea that resiliency skills can be learned and taught in school ([<reflink idref="bib36" id="ref7">36</reflink>]; see <ulink href="http://strongkids.uoregon.edu/">http://strongkids.uoregon.edu/</ulink> for more information). Based on instructional design principles such as activating prior knowledge and providing opportunities for practice, Strong Kids was built on Cowen's (1994) five proposed pathways to psychological wellness as a basis for designing specific curricular concepts. These pathways include (a) promoting early attachments, (b) building competence with developmentally appropriate skills, (c) being in a setting that promotes wellness, (d) feeling empowered about the future, and (e) coping with stress using relevant skills. There are 12 lessons that comprise the program and each lesson is typically taught in a 45- to 50-min session. Techniques found in cognitive-behavioral therapeutic approaches are integrated throughout many of the lessons, such as emotional identification, challenging negative and maladaptive thoughts, relaxation strategies, goal setting, and learning to think with optimism. Although prior pilot research has tentatively established the effectiveness of Strong Kids with students in the elementary-level general education environment and in some special education settings ([<reflink idref="bib33" id="ref8">33</reflink>]; [<reflink idref="bib57" id="ref9">57</reflink>]), more evidence is needed to determine its effectiveness with middle school-age students; this was a primary aim of this study.</p> <p>Simply implementing an SEL program is not enough. The quality or the way in which a program is implemented is also known to promote social and emotional development ([<reflink idref="bib48" id="ref10">48</reflink>]). CASEL (2003) proposed three qualities to implementation that have particular importance and pertinence to this study: (a) SEL programs should provide teacher training as part of implementation, (b) ongoing technical assistance should be provided during implementation, and (c) implementation of the program should be monitored using data that may improve subsequent implementation efforts. Fixsen, Naoom, Blasé, Friedman, and Wallace (2005) agreed that these areas are fundamental to SEL implementation efforts so teachers and students may benefit. Undeniably, teachers are frequently asked to execute a program, often very new to the teacher and school, and do so with the expectation that students <emph>will</emph> demonstrate gains. Very often, teachers do not receive enough training before they begin instruction and adequate support during the instructional phase when many questions inevitably arise. This can lead to frustration and sometimes abdication from the very instructional methods and conditions under which the program was proven to be effective. Although some teachers obtain training via a part- or full-day workshop ([<reflink idref="bib26" id="ref11">26</reflink>]), this "train-and-hope" method ([<reflink idref="bib55" id="ref12">55</reflink>]) rarely provides the comprehensive support needed to affect treatment integrity ([<reflink idref="bib59" id="ref13">59</reflink>]) or students' skills ([<reflink idref="bib1" id="ref14">1</reflink>]; [<reflink idref="bib17" id="ref15">17</reflink>]; [<reflink idref="bib26" id="ref16">26</reflink>]; [<reflink idref="bib47" id="ref17">47</reflink>]).</p> <p>What method of technical assistance or support is appropriate and effective when applied to a social-emotional learning program? At this time, it is unclear as to the most effective approach that would not only be effective but also be accepted by school personnel implementing the program. To investigate feasible options for the purposes of the exploratory portion of this study, the consultation literature was reviewed to determine any viable strategies. Several consultation methodologies have been effectively implemented in a classroom setting, most notably those derived from mental health consultation, organizational consultation, and behavioral consultation, the latter which is possibly the most recognized in the school-based consultation literature (see [<reflink idref="bib53" id="ref18">53</reflink>], and [<reflink idref="bib24" id="ref19">24</reflink>], for this discussion). In our search for an effective and feasible strategy, research pertaining to school-based, health promotion programs that utilized consultation suggested that a behaviorally oriented process that resembled performance feedback was typically used ([<reflink idref="bib5" id="ref20">5</reflink>]; [<reflink idref="bib11" id="ref21">11</reflink>]; [<reflink idref="bib15" id="ref22">15</reflink>]; [<reflink idref="bib50" id="ref23">50</reflink>]; [<reflink idref="bib54" id="ref24">54</reflink>]). Results of these studies revealed that technical support using a performance feedback process facilitated quality implementation of the program and beneficial results to participants (see [<reflink idref="bib48" id="ref25">48</reflink>], for a comprehensive summary).</p> <p>Performance feedback is particularly effective at improving treatment integrity of the individual providing a service or treatment as well as student outcomes. The organizational behavior and management literature has demonstrated that employee performance in the workplace can improve by using consultants to deliver performance feedback to the employee ([<reflink idref="bib2" id="ref26">2</reflink>]; [<reflink idref="bib3" id="ref27">3</reflink>]). In the school-based literature, Cossairt, Hall, and Hopkins (1973) demonstrated a functional relationship between teacher behavior and experimenter feedback, suggesting that feedback can alter teacher behavior. In recent years, performance feedback has been used to improve teachers' adherence to classroom interventions and improve student behavioral and academic performance ([<reflink idref="bib41" id="ref28">41</reflink>]; [<reflink idref="bib43" id="ref29">43</reflink>]; [<reflink idref="bib45" id="ref30">45</reflink>]; [<reflink idref="bib46" id="ref31">46</reflink>]; [<reflink idref="bib47" id="ref32">47</reflink>]). The use of a consultant to provide support to teachers has produced increased adherence to program implementation and improved students' academic and behavioral skills ([<reflink idref="bib18" id="ref33">18</reflink>]; [<reflink idref="bib19" id="ref34">19</reflink>]; [<reflink idref="bib45" id="ref35">45</reflink>]; [<reflink idref="bib46" id="ref36">46</reflink>]). Performance feedback utilizes a consultant to present the implementer of a treatment, generally a teacher, with information/ data on observed performance (e.g., integrity to treatment), responds to any questions the teacher may have, and provides recommendations that should improve performance ([<reflink idref="bib46" id="ref37">46</reflink>]). Data are collected to track implementation integrity as well as to illustrate this performance to the teacher ([<reflink idref="bib43" id="ref38">43</reflink>]). These steps are compared and contrasted with other consultation models in Table 1.</p> <p>Given the evidence demonstrating the effectiveness of performance feedback on improving treatment implementation and student outcomes, the application of performance feedback to promote the effective use of a promising SEL program used with general education students appears most relevant and plausible. In sum, performance feedback was chosen for this study because (a) it has been used successfully in classroom settings with teachers and effecting student change, (b) integrity data was planned to be collected to verify implementation of the Strong Kids program and it could be easily used as a part of the performance feedback, and (c) providing feedback takes about 15 min, a reasonable amount of time to spend as a practitioner during a busy day. Table 1 illustrates the portions of performance feedback from other studies that were used to construct the performance feedback procedure for this study.</p> <p>TABLE 1 Components of Consultation Methods</p> <p> <ephtml> <table><thead valign="bottom"><tr><td valign="bottom">Consultation method</td><td valign="bottom">Characteristics</td><td valign="bottom">Components</td><td valign="bottom">Selected citation(s)</td></tr></thead><tbody><tr><td valign="top">Performance feedback</td><td valign="top"><list list-type="Bullet"><list-item><p>• <bold>Consultation extends beyond didactics</bold></p></list-item><list-item><p>• Practicing skills prior to implementation is helpful</p></list-item><list-item><p>• <bold>Prompts, feedback, self-management strategies contribute to interventionist behaviors</bold></p></list-item></list></td><td valign="top"><list list-type="Bullet"><list-item><p>• Observation of implementation skills</p></list-item><list-item><p>• Collect integrity data</p></list-item><list-item><p>• Review data with consultee</p></list-item><list-item><p>• Identify treatment steps missed</p></list-item><list-item><p>• Consultant suggests areas for improvements</p></list-item></list></td><td valign="top">Harchick, Sherman, Sheldon, & Strouse (1992); Noell, Duhon, Gatti, & Connell (2002)</td></tr><tr><td valign="top">Problem-solving consultation</td><td valign="top"><list list-type="Bullet"><list-item><p>• <bold>Consultant facilitates process</bold></p></list-item><list-item><p>• <bold>Indirect service delivery</bold></p></list-item><list-item><p>• <bold>Consultee is jointly involved in process</bold></p></list-item></list></td><td valign="top"><list list-type="Bullet"><list-item><p>• Elicit description of problem</p></list-item><list-item><p>• Assist in analyzing the problem</p></list-item><list-item><p>• Coconstruct interventions plan</p></list-item><list-item><p>• Monitor the program during implementation</p></list-item></list></td><td valign="top">Kratochwill, Elliott, & Callan-Stoiber (2002)</td></tr><tr><td valign="top">Motivational interviewing</td><td valign="top"><list list-type="Bullet"><list-item><p>• <bold>Honors client's expertise</bold></p></list-item><list-item><p>• Resources and motivation for change reside within client</p></list-item><list-item><p>• <bold>Affirms client right to self-direction; supports informed choice</bold></p></list-item></list></td><td valign="top"><list list-type="Bullet"><list-item><p>• <bold>F</bold>: Provide data-based feedback on behavior</p></list-item><list-item><p>• <bold>R</bold>: Clients have responsibility for behavior change</p></list-item><list-item><p>• <bold>A</bold>: Advice from expert is needed/provided</p></list-item><list-item><p>• <bold>M</bold>: Menu of options is developed with client</p></list-item><list-item><p>• <bold>E</bold>: Empathy is expressed for client's situation</p></list-item><list-item><p>• <bold>S</bold>: Self-efficacy is facilitated via client participation</p></list-item></list></td><td valign="top"><xref ref-type="bibr" rid="bibr40">Miller & Rollnick (2002)</xref>; Dishion & Kavanagh (2003)</td></tr><tr><td valign="top"><bold>Process for enhanced performance feedback</bold></td><td valign="top">See bolded items</td><td valign="top"><list list-type="Bullet"><list-item><p>• Observe teaching Strong Kids lessons</p></list-item><list-item><p>• Collect integrity data and graph</p></list-item><list-item><p>• Review data with teacher including components that were performed with and without integrity</p></list-item><list-item><p>• Three praise statements provided</p></list-item><list-item><p>• Teacher-initiated menu of options generated regarding instructional changes to improve integrity</p></list-item><list-item><p>• Discuss whether any barriers present or resources needed</p></list-item><list-item><p>• Area for change agreed upon by consultant and teacher</p></list-item></list></td><td valign="top">See aforementioned citations</td></tr><tr><td valign="top"><italic>Note</italic>. Bolded items = the components of consultation methods that influenced the consultation process used for this study.</td></tr></tbody></table> </ephtml> </p> <p>The exact protocol by which performance feedback is conducted varies and determining a standardized methodology was prioritized for this study. Various descriptions of the process are available such as a "discussion" between teacher and consultant ([<reflink idref="bib41" id="ref39">41</reflink>]), a one-one consulting process ([<reflink idref="bib22" id="ref40">22</reflink>]), or simply documenting the steps involved in the performance feedback process: data are reviewed with the consultee, areas where treatment was not implemented are discussed, and the consultant suggests areas for improvement ([<reflink idref="bib43" id="ref41">43</reflink>]). Exactly how this feedback is delivered is important. Communication during performance feedback has been described as an evolution of the problem-solving process ([<reflink idref="bib47" id="ref42">47</reflink>]), which presumably applies a collaborative approach found in school-based consultation procedures ([<reflink idref="bib30" id="ref43">30</reflink>]). The consultee is expected to participate and contribute to the problem analysis and plan construction phases rather than passively receive feedback on performance and implement as directed by the consultant ([<reflink idref="bib30" id="ref44">30</reflink>]).</p> <p>An additional consultation process we believed important to investigate for this study is motivational interviewing (MI), having gained wide acceptance for its effectiveness with individuals with alcohol consumption habits deemed problematic ([<reflink idref="bib6" id="ref45">6</reflink>]). A key element of motivational interviewing that was of particular interest is the attention that a consultant pays to increasing a consultee's motivation to change attitudes and behavior by actively discussing any barriers that may interfere with making changes that may ultimately improve program implementation and subsequent student performance. In brief, we believed this to be a vital component of the school-based consultation process that has not received enough attention to date. [<reflink idref="bib4" id="ref46">4</reflink>] discussed the potential of using MI as a complementary addition to a problem-solving model of consultation by addressing consultees' resistance and ambivalence to change. Simply stated, a consultant can use performance feedback by providing information on a consultee's performance and offer suggestions for change, but this does not acknowledge or address a consultee's resistance or ambivalence to change, especially when asked to implement new and unfamiliar classroom material, such as an SEL program. Current school-based consultation approaches either presume that a consultee is ready to make changes or that any resistance to change will naturally be addressed due to active participation in the process (see [<reflink idref="bib4" id="ref47">4</reflink>], for this discussion). For those consultees who are recipients of performance feedback, it may prove beneficial to consider <emph>and</emph> actively consider resistance-to-change issues.</p> <p>The use of MI methodology in conjunction with performance feedback was piloted by Reinke, Lewis-Palmer, and [<reflink idref="bib33" id="ref48">33</reflink>], whereby a consultant reviewed performance data with a teacher (consultee) with the goal of improving student outcomes. In this study, the consultant facilitated a process to collaboratively generate a "menu of options" regarding areas that might be targeted for change with specific attention to the consultee's preferences. Reinke et al.'s process recognized the need for the consultation interaction to be supportive, consultee-centered, and that it is the consultee who can make the changes that are not due to external pressures or ultimatums to change existing habits. Combining a component of motivational interviewing (i.e., generating a menu of options in a manner that is client driven and clinician supportive) with performance feedback provided a discriminative addition to the communication process by which performance feedback is delivered. This study wished to build upon Reinke et al.'s initial investigation by again using performance feedback in conjunction with keys elements of motivational interviewing as a means to ensure an SEL program was delivered with integrity in order to promote positive student outcomes. For the purpose of brevity, we referred to this performance feedback, motivational interviewing combination as "enhanced performance feedback" and recognize that the use of this methodology is exploratory in this study. Although we did not specifically measure whether resistance to change was lessened as a result of using enhanced performance feedback, the use of components of motivational interviewing known to support this effort was believed to lead to positive implementation integrity. Table 1 also illustrates the components of motivational interviewing that were used to construct enhanced performance feedback.</p> <p>Finally, recent evidence has indicated that teachers who engaged in a performance feedback process found it to be supportive and positive ([<reflink idref="bib43" id="ref49">43</reflink>]). Additionally, the extent to which teachers correctly implement an intervention is directly related to their belief that the program is not only easy to implement but also will lead to positive results ([<reflink idref="bib56" id="ref50">56</reflink>]). Social validity, or how an intervention is viewed as acceptable, relevant, and viable ([<reflink idref="bib51" id="ref51">51</reflink>]), is vital to the success of an SEL program because the likelihood of the program being used as intended is dependent on how it is perceived. Strong Kids was designed to be easy to implement and more data are needed to determine the extent to which implementers of the program agree.</p> <p>In sum, the purpose of this study was to (a) evaluate whether students could obtain knowledge of skills that may be used to boost mental health immunity and ease mental health symptoms, if present; (b) continue our research on the effectiveness of the Strong Kids program, especially with sixth-grade students, regarding knowledge of social-emotional skills students may acquire after participating in the program, as well as any change in internalizing symptoms they may experience; (c) evaluate whether student outcomes are positively impacted when students participate in the Strong Kids program with a teacher who has participated in an enhanced performance feedback process; and (d) explore teachers' opinions regarding the Strong Kids program and participation in the enhanced performance feedback process.</p> <hd id="AN0059165621-3">METHOD</hd> <p></p> <hd id="AN0059165621-4">Design</hd> <p>A quasi-experimental design with a no treatment control group was used for this study. Student participants had already been assigned at the start of the school year to Language Arts Block classes and although ostensibly assigned to these classes in a somewhat random manner, this factor led to the use of a quasi-experimental procedure rather than true random assignment. The two independent variables for this study were (a) <emph>Time</emph>, with two levels: pretest and posttest, and (b) <emph>Group</emph>, with three levels: a comparison group, a group of students who were taught the Strong Kids program and whose teacher received enhanced performance feedback, and a group of students who were also taught the Strong Kids program and whose teacher did not receive enhanced performance feedback.</p> <hd id="AN0059165621-5">Participants</hd> <p>Participants for this study included 139 eligible sixth-grade students selected from existing general education language arts classrooms in a midsize community (mean age = 11.5 years; boys = 71, girls = 57). Out of 139 students who were eligible, 10% of these students (<emph>n</emph> = 14) either elected to not participate in the study, their parents indicated they did not want their child to participate, or the student had incomplete data due to absence or withdrawal from the school at either pretest or posttest, yielding a final student participant total of 125. Students who received special education services in the area of language arts did not participate as they typically received small group instruction during this class period. Demographic data for this middle school indicated that 33.3% of the students qualified for free or reduced-price lunch, with 90% of the students described as White, 2% as Black, 3% Hispanic, 3% as American Indian, 1% as Asian, and 1% as Unspecified. The middle school elected to adopt the Strong Kids program as part of the language arts program, and all students enrolled in sixth-grade, general education language arts received the instructional content. The 3 language arts teachers who provided instruction to these students also participated in the study, 1 teacher in the control group and 2 in intervention conditions. The Strong Kids program was taught in general education classrooms during a 50-min time period when language arts instruction was typically conducted. The principal investigator for this study served as the consultant to the teacher in the "Enhanced Performance Feedback" intervention group.</p> <hd id="AN0059165621-6">Procedures</hd> <p>Participants were recruited via correspondence with the school districts' research compliance officer and the principal of the middle school and were selected based on the teachers' interest and ability to participate. Students who elected, either self-elected or by parent request, to not participate in data collection procedures were supervised in a study activity with typical homework or reading materials.</p> <p>One week prior to teachers' implementing the Strong Kids program, the 2 teachers in the intervention groups participated in a 2-hr training. The 3rd teacher, who taught students serving in the control group, attended the training briefly to review consent and data collection dates and procedures. The purpose of the training was to provide general information regarding the organization of the curricular lessons, arrange for the materials needed to implement the program, and discuss specific areas that may be particularly challenging. Following the training, the intervention group teachers were randomly assigned to one of two intervention conditions: a "Standard Instruction" intervention group where the teacher did not receive any support from the consultant for the duration of program implementation and an "Enhanced Performance Feedback" intervention group where the teacher received regular, enhanced performance feedback and support from the consultant throughout the course of program implementation. Randomization to control and intervention groups was not possible due to one of the teachers being on a personal leave during the recruitment phase, which led to this teacher-student grouping to be assigned to the control group. A "Frequently Asked Questions" handout was provided to the teacher in the "Standard Instruction" group with basic information and highlighting the importance of contacting the consultant should questions or an emergency arise. There was no contact between the consultant and this teacher for the duration of program implementation.</p> <p>The teachers in the intervention groups varied somewhat in the number of years they have taught (12 vs. 5.5 years) and educational degrees (bachelor's vs. master's) but were similar in that they both had experience teaching a social and emotional learning program in the past, <emph>Second Steps</emph>, and had limited experience with receiving consultation. Table 2 illustrates this demographic information. Teachers in both intervention groups implemented the Strong Kids program on the same day of the week, once per week, with the exception of spring break recess. Each lesson was taught during a 50-min time block reserved for usual language arts instruction. This study was limited by the use of only 2 teachers, 1 teacher receiving enhanced performance feedback and 1 not, and comparing student outcomes across these two groups. However, the exploratory nature of this research rendered this flaw temporarily acceptable and posed possibility for future research should this approach appear tentatively viable.</p> <p>TABLE 2 Teachers' Demographic Information Regarding Their Experience</p> <p> <ephtml> <table><thead valign="bottom"><tr><td valign="bottom">Demographic</td><td valign="bottom">Standard instruction</td><td valign="bottom">Enhanced performance feedback</td></tr></thead><tbody><tr><td valign="top">Years teaching</td><td valign="top">12</td><td char="." valign="top">5.5</td></tr><tr><td valign="top">Educational degree</td><td valign="top">Bachelor of Arts</td><td valign="top">Master of Arts</td></tr><tr><td valign="top">Additional credentials</td><td valign="top">Social Studies</td><td valign="top">—</td></tr><tr><td valign="top">Prior experience teaching an SEL program</td><td valign="top">Second Steps</td><td valign="top">Second Steps</td></tr><tr><td valign="top">Past experience receiving consultation support</td><td valign="top">Workshop on Second Steps</td><td valign="top">Mentors</td></tr><tr><td valign="top"><italic>Note</italic>. SEL = social-emotional learning.</td></tr></tbody></table> </ephtml> </p> <p>Measurement of intervention integrity was used (a) to determine the extent to which the program was implemented as designed by both teachers in the intervention groups, and (b) as a component of the enhanced performance feedback procedure. In the absence of enhanced performance feedback and under typical programmatic implementation procedures, measuring intervention integrity leads to a more robust probability that the data are representative of the effects of the intervention ([<reflink idref="bib20" id="ref52">20</reflink>]; [<reflink idref="bib56" id="ref53">56</reflink>]). Specifically, it is imperative to understand the extent to which the lessons in the program were implemented as designed and that enhanced performance feedback was delivered as designed based on theoretical and evidence bases. Additionally, there is considerable value in monitoring implementation procedures as failing to do so may lead to serious misunderstandings regarding the reasons for programmatic "failure" or poor student outcomes ([<reflink idref="bib23" id="ref54">23</reflink>]). In other words, if prevention and intervention programs have established effectiveness under laboratory conditions, which typically use integrity measurements, it is better to fund and support these same or grossly similar practices under real-world circumstances than to discontinue the use of an effective program that is used incorrectly or suboptimally.</p> <p>Treatment integrity data were collected via direct observation by research assistants for both teachers in the intervention groups during 7 of the 12 lessons that were implemented (58% of the lessons). Checklists containing the primary components of these lessons were used in conjunction with direct observation of the teachers to gather intervention integrity data. Teachers were not told the dates on which the observations would be taking place but had been informed to expect observations to occur randomly. Teachers' performance was rated on a 3-point scale as having "Not Implemented," "Partially Implemented," or "Fully Implemented" each primary component. A percentage ratio was calculated based on the number of components fully implemented compared with the components that should have been fully implemented (i.e., all components). These data were plotted on a graph and used as a visual aid when providing enhanced performance feedback to the teacher in the "Enhanced Performance Feedback" intervention group.</p> <p>Interobserver reliability was assessed by additional research assistants for four of the seven lessons where intervention integrity data were collected. Over the course of all lessons in which interobserver data were collected, interobserver agreement was at or above 94% for both intervention conditions. This finding indicates that data collectors were in high agreement in their observations of the extent to which teachers were fully implementing key components of each lesson.</p> <p>The teacher in the "Enhanced Performance Feedback" condition participated in regular, enhanced performance feedback sessions. Feedback occurred on the same day in which intervention integrity data were collected and was conducted without the teacher knowing ahead of time the day in which these data would be collected and feedback delivered. This procedure was done to increase the likelihood that the teacher would engage in similar lesson preparation activities throughout the entire course of lesson implementation, avoiding extra preparation on observation days and consequently inflating performance data. Intervention integrity data were collected and feedback was given after every lesson instruction during Lessons 1 through 5, then given for Lessons 8 and 11 (7 out of 12 lessons or 57% of lessons implemented). This interval was (a) used to give the teacher more support during the initial lessons when more support may be needed, and (b) believed to reasonably represent the amount of time school practitioners would be able to allocate to this activity (approximately 1 hr of time on seven occasions). Additionally, given prior studies' use of daily performance feedback when an intervention was implemented daily ([<reflink idref="bib59" id="ref55">59</reflink>]) and provided weekly ([<reflink idref="bib41" id="ref56">41</reflink>]) that both led to positive outcomes for students, the use of intermittent feedback at a rate of 57% for instruction delivered once per week seemed reasonable to test the hypothesis that enhanced performance feedback would lead to positive student outcomes.</p> <p>Immediately following instruction, the consultant met with the teacher and reviewed the intervention integrity data, a procedure by which consultees are provided with a visual aid to view their performance ([<reflink idref="bib3" id="ref57">3</reflink>]; [<reflink idref="bib41" id="ref58">41</reflink>]; [<reflink idref="bib45" id="ref59">45</reflink>]; [<reflink idref="bib46" id="ref60">46</reflink>]; [<reflink idref="bib47" id="ref61">47</reflink>]). The percentage of components implemented as well as the particular components that were fully, partially, and not implemented were reviewed. The motivational interviewing component of the enhanced performance feedback procedure was then used whereby a minimum of three praise statements were given to the teacher regarding instructional performance. The consultant prompted the teacher to generate ideas where improvements might be made based on the teacher's personal preferences and feasibility. The final decision regarding the area in which improvement would be sought was based on the teacher's preferences. A self-report checklist was used to ensure all components in the enhanced performance feedback procedure were implemented by the consultant. These data indicated that the components of each enhanced performance feedback procedure were completed at a rate of 100% and verified with interrater reliability observations by an independent observer. The interrater reliability calculations for four of the seven feedback sessions were measured to be at 100%. In addition to the seven feedback sessions, the consultant called or e-mailed the teacher to check in and determine whether additional support was desired. The specific components of the enhanced performance feedback procedure and examples of content from this procedure are illustrated in Table 3.</p> <p>Students in the intervention and control groups were asked to complete self-report questionnaires (student outcomes measures) at two times: the first, on the day instruction began for the intervention groups with Lesson 1, but before the students received any content from this lesson, and the second, following the conclusion of Lesson 12, after the students in the intervention groups had received all of the content from the program. During this time, the teachers in the intervention groups completed a social validity survey.</p> <hd id="AN0059165621-7">Measures</hd> <p></p> <hd id="AN0059165621-8">Student knowledge of social and emotional coping strategies</hd> <p>A 20-item, self-report questionnaire comprised of true-false and multiple-choice questions was used to measure students' knowledge of social and emotional coping strategies. This measure was developed specifically for the evaluation of the Strong Kids program and has been used in several related studies (e.g., [<reflink idref="bib33" id="ref62">33</reflink>]; [<reflink idref="bib42" id="ref63">42</reflink>]; [<reflink idref="bib57" id="ref64">57</reflink>]). Questions represent key content areas of the program, for example, recognizing primary vocabulary terms and applying problem-solving models used throughout the curriculum to a described situation. Internal consistency alpha coefficients from previous studies have yielded reliability estimates between.60 and.70, which is considered adequate for a brief research tool not used for individual student decision making. This instrument is available on the Strong Kids Web site at <ulink href="http://strongkids.uoregon.edu">http://strongkids.uoregon.edu</ulink></p> <hd id="AN0059165621-9">Symptoms of depression and anxiety</hd> <p>Two self-report measures were used to assess students' symptoms of depression and anxiety: the Children's Depression Inventory (CDI; Kovacs, 1992) and the Internalizing Symptoms Scale for Children (ISSC; Merrell & Walters, 1998). The CDI is a 27-item questionnaire that measures negative mood, interpersonal problems, feelings of ineffectiveness and anhedonia, and negative self-esteem. Three response choices are provided for each item and the respondent must choose the response that best describes his or her thoughts and behaviors in the past 2 weeks. Internal consistency for the CDI is strong (.80s) and its test-retest reliability for short and long time intervals ranges between.70 and.85. The CDI has been used extensively in research as a self-report measure for children and correlates highly with other measurements of internalizing symptoms ([<reflink idref="bib33" id="ref65">33</reflink>]). The ISSC is a 48-item self-report assessment instrument that measures cognitive, affective, and behavioral traits associated with depression and anxiety and is appropriate for students in Grades 3 to 6. Two factor scores measuring positive affect and negative affect/general distress are yielded in addition to a total symptoms score. Examples of items measuring positive affect include "I feel cheerful" and "I feel important." Examples of items measuring negative affect/general distress are "I worry about things" and "I feel lonely." Respondents were asked to consider how the 48 items represent their thoughts and feelings on a 4-point Likert scale system from <emph>never</emph> to <emph>often</emph>. Internal consistency is very high for this measure (.90 and higher) and test-retest reliability at short and medium time intervals is strong (.70 to.80). The ISSC correlates very well with other self-report measures including the CDI, Youth Self-Report, Revised Children's Manifest Anxiety Scale, and Behavioral Assessment System for Children ([<reflink idref="bib34" id="ref66">34</reflink>]). The ISSC has demonstrated differentiation between children referred for internalizing and noninternalizing problems and significantly correlates with broadband scores on the Child Behavior Checklist ([<reflink idref="bib38" id="ref67">38</reflink>]).</p> <p>TABLE 3 Consultation Procedures Checklist: Examples From Lessons 1, 2, and 3</p> <p> <ephtml> <table><thead valign="bottom"><tr><td valign="bottom">Lesson</td><td valign="bottom">Date</td><td valign="bottom">PF Given?</td><td valign="bottom">Reviewed Integrity Data?</td><td valign="bottom">Graph Data</td><td valign="bottom">Review Graph w/Teacher</td><td valign="bottom">3 Praise Statements Given</td><td valign="bottom">Menu of Options Discussed</td><td valign="bottom">Resources Needed/Barriers Discussed</td><td valign="bottom">Goal Agreed Upon</td><td valign="bottom">Goal Is:</td></tr><tr><td /><td /><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td valign="bottom">Y/N</td><td /></tr></thead><tbody><tr><td valign="top">1</td><td valign="top">2/16</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y: Reduce student examples giving; improve pace, fluency of delivery</td><td valign="top">Y: Review lessons ahead of time</td><td valign="top">Y</td><td valign="top">Focus on delivering content vs. adhering to scripts and remaining flexible</td></tr><tr><td valign="top">2</td><td valign="top">2/23</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y: Use bulleted points rather than the scripts; preview scripts, picking out important points</td><td valign="top">Y: Review lessons ahead of time to pick out important points</td><td valign="top">Y</td><td valign="top">Use scripts flexibly, picking out main points</td></tr><tr><td valign="top">3</td><td valign="top">3/2</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y</td><td valign="top">Y: Increase processing content with students; use more generic examples—relevant to most students</td><td valign="top">N</td><td valign="top">Y</td><td valign="top">Use more generic examples but relevant to most students</td></tr><tr><td valign="top"><italic>Note</italic>. PF = performance feedback.</td></tr></tbody></table> </ephtml> </p> <hd id="AN0059165621-10">Teacher attitudes toward the program</hd> <p>The Strong Kids program is a new program for most teachers as it continues to be evaluated on the basis of its effectiveness <emph>and</emph> the ease with which it may be integrated into a general education classroom and taught by teachers. The likelihood of a program producing positive outcomes for students also depends on whether the procedures used are acceptable and feasible to those charged with implementation, namely, teachers ([<reflink idref="bib21" id="ref68">21</reflink>]). In this study, teachers' attitudes toward the Strong Kids program was measured using a 32-item social validity questionnaire developed specifically for this project. This measure generated qualitative responses and was completed by the two teachers who implemented the program after they taught the last lesson (Lesson 12). Questions were structured around five domains of inquiry: (a) How similar are teachers' goals to those of the program? (b) How acceptable are the procedures that were used to implement the program? (c) How satisfied were teachers with any effects they observed on student knowledge and behaviors? (d) How feasible was it to implement the program? (e) How confident were teachers in their skills to implement the program? Teachers were also provided an opportunity to disclose additional information they believed valuable and important to their views and experiences. For each item, respondents chose one of three response choices. For example, for the question "How acceptable was the amount of time it took to implement each lesson?" the respondent chose one response choice from the following three: <emph>not acceptable</emph>, <emph>somewhat acceptable</emph>, <emph>very acceptable</emph>.</p> <p>TABLE 4 Descriptive Statistics for Outcome Variables at Pretest and Posttest</p> <p> <ephtml> <table><thead valign="bottom"><tr><td valign="bottom">Measure</td><td valign="bottom">Control <italic>n</italic> = 46</td><td valign="bottom">Standard instruction <italic>n</italic> = 40</td><td valign="bottom">Enhanced PF <italic>n</italic> = 39</td></tr><tr><td valign="bottom"><italic>M</italic></td><td valign="bottom"><italic>SD</italic></td><td valign="bottom"><italic>M</italic></td><td valign="bottom"><italic>SD</italic></td><td valign="bottom"><italic>M</italic></td><td valign="bottom"><italic>SD</italic></td></tr></thead><tbody><tr><td valign="top">Knowledge</td><td /><td /><td /><td /><td /><td /></tr><tr><td valign="top">      Pretest</td><td char="." valign="top">12.14</td><td char="." valign="top">3.51</td><td char="." valign="top">12.73</td><td char="." valign="top">3.45</td><td char="." valign="top">14.22</td><td char="." valign="top">3.32</td></tr><tr><td valign="top">      Posttest</td><td char="." valign="top">12.36</td><td char="." valign="top">3.11</td><td char="." valign="top">15.60</td><td char="." valign="top">2.99</td><td char="." valign="top">16.15</td><td char="." valign="top">2.64</td></tr><tr><td valign="top">ISSC</td><td /><td /><td /><td /><td /><td /></tr><tr><td valign="top">      Pretest</td><td char="." valign="top">45.04</td><td char="." valign="top">19.26</td><td char="." valign="top">44.85</td><td char="." valign="top">26.96</td><td char="." valign="top">45.80</td><td char="." valign="top">23.39</td></tr><tr><td valign="top">      Posttest</td><td char="." valign="top">45.11</td><td char="." valign="top">20.64</td><td char="." valign="top">39.73</td><td char="." valign="top">24.75</td><td char="." valign="top">45.24</td><td char="." valign="top">22.34</td></tr><tr><td valign="top">ISSC Factor 1</td><td /><td /><td /><td /><td /><td /></tr><tr><td valign="top">      Pretest</td><td char="." valign="top">38.26</td><td char="." valign="top">16.32</td><td char="." valign="top">37.60</td><td char="." valign="top">20.97</td><td char="." valign="top">38.59</td><td char="." valign="top">19.66</td></tr><tr><td valign="top">      Posttest</td><td char="." valign="top">38.28</td><td char="." valign="top">16.41</td><td char="." valign="top">33.80</td><td char="." valign="top">18.67</td><td char="." valign="top">37.69</td><td char="." valign="top">18.63</td></tr><tr><td valign="top">ISSC Factor 2</td><td /><td /><td /><td /><td /><td /></tr><tr><td valign="top">      Pretest</td><td char="." valign="top">8.85</td><td char="." valign="top">6.86</td><td char="." valign="top">9.85</td><td char="." valign="top">9.74</td><td char="." valign="top">9.74</td><td char="." valign="top">9.14</td></tr><tr><td valign="top">      Posttest</td><td char="." valign="top">9.33</td><td char="." valign="top">8.12</td><td char="." valign="top">8.05</td><td char="." valign="top">9.66</td><td char="." valign="top">8.77</td><td char="." valign="top">7.65</td></tr><tr><td valign="top">CDI</td><td /><td /><td /><td /><td /><td /></tr><tr><td valign="top">      Pretest</td><td char="." valign="top">5.87</td><td char="." valign="top">5.67</td><td char="." valign="top">6.82</td><td char="." valign="top">9.41</td><td char="." valign="top">6.00</td><td char="." valign="top">7.41</td></tr><tr><td valign="top">      Posttest</td><td char="." valign="top">7.36</td><td char="." valign="top">9.05</td><td char="." valign="top">6.50</td><td char="." valign="top">9.36</td><td char="." valign="top">6.00</td><td char="." valign="top">5.89</td></tr><tr><td valign="top"><italic>Note</italic>. PF = performance feedback; ISSC = Internalizing Symptoms Scale for Children; CDI = Children's Depression Inventory.</td></tr></tbody></table> </ephtml> </p> <hd id="AN0059165621-11">RESULTS</hd> <p>Descriptive statistics for the various outcome measures across the three study groups over time were calculated. These results are presented in Table 4. Student outcomes are discussed in the first part of the Results section; teacher outcomes are discussed in the latter part.</p> <hd id="AN0059165621-12">Student Outcomes</hd> <p></p> <hd id="AN0059165621-13">Knowledge of healthy social-emotional behavior</hd> <p>The first goal of this study was to determine whether the Strong Kids program would have an impact on students' knowledge of healthy social-emotional behavior. A two-way repeated measures analysis of variance (ANOVA) was conducted to evaluate this question. Power of the analysis may have been diminished by using an ANOVA rather than an analysis of covariance (ANCOVA) because having nonequivalent groups (i.e., using intact classroom groups rather than randomly assigning students to treatment groups) violates one of the assumptions of an ANCOVA and interpretation of results may subsequently be questionable due to the probability of differences already occurring between nonequivalent groups at pretest ([<reflink idref="bib28" id="ref69">28</reflink>]). The dependent variable for the analysis was knowledge of skills as measured by the Strong Kids <emph>Knowledge Test</emph>. The within-subjects variable was Time at pretest and posttest. The between-subjects variable was Group (Control, Enhanced Performance Feedback, Standard Instruction). The Time by Group interaction and main effects were tested using the multivariate criterion of Wilks's Lambda (Λ). As hypothesized, a significant interaction between Group and Time was detected, Λ =.85, <emph>F</emph>(<reflink idref="bib1" id="ref70">1</reflink>, 122) = 10.89, <emph>p</emph> <.01, indicating a difference in knowledge between groups and across time. Observed power using Wilks's Lambda was.99, <emph>p</emph> =.05, suggesting a very low chance of making a Type II error. Post hoc pairwise analyses using the Bonferroni method to control for Type I error indicated that the difference in knowledge scores between the Standard Instruction group and the Control group was significant (−1.91, <emph>p</emph> =.01) as was the difference in knowledge between the Control group and the Standard Instruction group (−2.93, <emph>p</emph> = <.01). These findings indicate that students who participated in the Strong Kids program demonstrated significantly more knowledge of healthy social-emotional behaviors after participating in the program than those students who did not receive this instruction.</p> <hd id="AN0059165621-14">Internalizing symptoms</hd> <p>The second goal of this study was to evaluate whether participation in Strong Kids could lead to decreases in internalizing symptoms. A two-way repeated measures ANOVA was conducted for data collected with the CDI and ISSC. Time and Group interaction and main effects were tested using the Wilks's Lambda criterion procedure. No significant interaction between Time and Group was detected for the CDI scores, Λ =.97, <emph>F</emph>(<reflink idref="bib2" id="ref71">2</reflink>, 123) = 1.65, <emph>p</emph> =.20, indicating no difference among the three groups over time and subsequently no change in internalizing symptoms as measured by this tool. Power to detect an effect suggested a moderate possibility of making a Type II error and not detecting an effect (.34, <emph>p</emph> =.05). No main effects for Group <emph>F</emph>(<reflink idref="bib2" id="ref72">2</reflink>, 123) =.10, <emph>p</emph> =.91 or Time <emph>F</emph>(<reflink idref="bib1" id="ref73">1</reflink>, 123) =.77, <emph>p</emph> =.38 were detected. Observed power for Group (.07, <emph>p</emph> =.05) and Time (.14, <emph>p</emph> =.05) were both low, indicating the possibility of not detecting an effect when one may exist. Likewise, no significant interaction between Time and Group was detected for ISSC scores, Λ =.97, <emph>F</emph>(<reflink idref="bib2" id="ref74">2</reflink>, 124) = 1.89, <emph>p</emph> =.16, indicating no significant differences in internalizing symptoms among the three groups over time as measured by the ISSC. Observed power was moderate (.39, <emph>p</emph> =.05). There were no significant main effects for Group <emph>F</emph>(<reflink idref="bib2" id="ref75">2</reflink>, 124) =.26, <emph>p</emph> =.77 and Time <emph>F</emph>(<reflink idref="bib1" id="ref76">1</reflink>, 124) = 2.53, <emph>p</emph> =.12. Observed power was low for Group (.09, <emph>p</emph> =.05) and moderate for Time (.35, <emph>p</emph> =.05).</p> <hd id="AN0059165621-15">Negative affect/General distress</hd> <p>In addition to a general internalizing symptoms score, the ISSC provides two factor scores to measure particular constructs associated with anxiety and depression. No significant interaction between Time and Group was detected for the <emph>Negative affect/General distress</emph> factor score, Λ =.98, <emph>F</emph>(<reflink idref="bib2" id="ref77">2</reflink>, 122) = 1.39, <emph>p</emph> =.25, indicating no difference among groups over time in this domain. Observed power was.29, <emph>p</emph> =.05. No significant main effects were found for Group <emph>F</emph>(<reflink idref="bib2" id="ref78">2</reflink>, 122) =.28, <emph>p</emph> =.76 and Time <emph>F</emph>(<reflink idref="bib1" id="ref79">1</reflink>, 122) = 2.53, <emph>p</emph> =.12. Power was low for Group (.09, <emph>p</emph> =.05) and moderate for Time (.31, <emph>p</emph> =.05). Likewise, for the Positive Affect scores, the ANOVA analysis yielded no significant interaction between Group and Time, Λ =.99, <emph>F</emph>(<reflink idref="bib2" id="ref80">2</reflink>, 122) = 2.04, <emph>p</emph> =.13. Observed power was.41, <emph>p</emph> =.05. Similarly, there were no main effects detected for Group <emph>F</emph>(<reflink idref="bib2" id="ref81">2</reflink>, 122) =.01, <emph>p</emph> =.99 or Time <emph>F</emph>(<reflink idref="bib1" id="ref82">1</reflink>, 122) = 2.58, <emph>p</emph> =.11. Observed power was low for Group (.05, <emph>p</emph> =.05) and moderate for Time (.36, <emph>p</emph> =.05).</p> <hd id="AN0059165621-16">Magnitude of treatment effects</hd> <p>In addition to the parametric statistical significance tests that were conducted, the standardized mean difference procedure ([<reflink idref="bib32" id="ref83">32</reflink>]) was used to evaluate the magnitude of the differences between measurements over time and among the groups. Specifically, although the ANOVA analyses overall did not yield significant findings, measuring magnitude of effects can be helpful when evaluating universally delivered interventions with a population sample that may not be experiencing many problem symptoms (see [<reflink idref="bib35" id="ref84">35</reflink>], for this discussion). Effect sizes (ES) were determined by calculating the difference in mean raw scores over time and between groups, the result divided by the harmonic standard deviation of the two groups that were compared. Effect sizes were interpreted using the guidelines recommended by [<reflink idref="bib9" id="ref85">9</reflink>] whereby an effect size of <.20 is not meaningful,.20–.49 is a meaningful but small effect,.50–.79 is a medium effect, and >.80 is considered a large effect. For this study, a positive effect valence was assigned when the effect was in the anticipated direction and a negative effect valence (e.g., −.65) was assigned when the effect was contrary to what was anticipated if the intervention proved to be effective. Results from this analysis are illustrated in Tables 4 and 5.</p> <p>Four meaningful positive effects were determined to occur over time from pretest to posttest across the three groups. These effects are illustrated in Table 5. In terms of knowledge of healthy social and emotional behaviors, a large effect was found for the Standard Instruction group indicating that students who received the program and whose teacher did not receive consultation support demonstrated meaningful and anticipated increases in their knowledge scores from pretest to posttest (ES =.89). Similarly, the Enhanced Performance Feedback group demonstrated a moderate, meaningful, and anticipated increase in knowledge (ES =.65) at the end program instruction. Regarding internalizing symptoms, a small and meaningful effect was demonstrated for the Control group (ES =.20) indicating an increase in symptoms at posttest as measured by the CDI. The same effect size (ES =.20) was calculated for the students in the Standard Instruction group indicating a decrease in symptoms at posttest as measured by the ISSC.</p> <p>TABLE 5 Magnitude of Effect Sizes From Pretest to Posttest for Dependent Measures</p> <p> <ephtml> <table><thead valign="bottom"><tr><td valign="bottom">Dependent measure</td><td valign="bottom">Control</td><td valign="bottom">Standard instruction</td><td valign="bottom">Enhanced PF</td></tr></thead><tbody><tr><td valign="top">Knowledge questionnaire</td><td char="." valign="top">.07</td><td char="." valign="top">.89<xref ref-type="fn" rid="fn1" /></td><td char="." valign="top">−.65<xref ref-type="fn" rid="fn1" /></td></tr><tr><td valign="top">ISSC total</td><td char="." valign="top">.00</td><td char="." valign="top">.20<xref ref-type="fn" rid="fn1" /></td><td char="." valign="top">.02</td></tr><tr><td valign="top">ISSC: Factor 1</td><td /><td /><td /></tr><tr><td valign="top">      Negative affect/General distress</td><td char="." valign="top">.00</td><td char="." valign="top">.19</td><td char="." valign="top">.05</td></tr><tr><td valign="top">ISSC: Factor 2</td><td /><td /><td /></tr><tr><td valign="top">      Positive affect</td><td char="." valign="top">.06</td><td char="." valign="top">.19</td><td char="." valign="top">.12</td></tr><tr><td valign="top">      CDI</td><td char="." valign="top">.20<xref ref-type="fn" rid="fn1" /></td><td char="." valign="top">.03</td><td valign="top">0</td></tr><tr><td valign="top">      ODR total</td><td char="." valign="top">−.28<xref ref-type="fn" rid="fn1" /></td><td char="." valign="top">.11</td><td char="." valign="top">.17</td></tr><tr><td valign="top"><italic>Note</italic>. PF = performance feedback; ISSC = Internalizing Symptoms Scale for Children; CDI = Children's Depression Inventory; ODR = office disciplinary referral.</td></tr><tr><td valign="top" /></tr></tbody></table> </ephtml> </p> <p>In terms of between-subjects effects, five meaningful positive effects were found among the groups when comparing posttest data and are illustrated in Table 6. The largest effects were found regarding knowledge of healthy social and emotional behaviors. When comparing the Control group with the intervention groups (Enhanced Performance Feedback, Standard Instruction) large effects were determined (ES = 1.32, 1.06, respectively) as anticipated indicating that the students who participated in the program demonstrated more knowledge than the Control group at posttest. Additionally, when comparing the intervention groups, the Enhanced Performance Feedback group had more knowledge at posttest than the Standard Instruction group (ES =.20). Regarding internalizing symptoms, the Standard Instruction group reported fewer symptoms than the Control group as measured by the ISSC (ES =.24) and an unanticipated, small and meaningful effect indicated the Enhanced Performance Feedback group had more symptoms than the Standard Instruction group (ES = −.23). This finding is interesting as it was hypothesized that when a teacher was given support while implementing the program, students would experience better outcomes when compared with the students whose teacher did not receive support, an assumption that did not prove to be true in this study.</p> <p>TABLE 6 Magnitude of Effect at Posttest Among Treatment Groups for Self-Report Measure (Total Scores as Measured by Effect Sizes)</p> <p> <ephtml> <table><thead valign="bottom"><tr><td /><td valign="bottom">Control</td><td valign="bottom">Standard instruction</td><td valign="bottom">Enhanced PF</td></tr><tr><td valign="bottom">Measure</td><td valign="bottom">ES</td><td valign="bottom">Measure</td><td valign="bottom">ES</td><td valign="bottom">Measure</td><td valign="bottom">ES</td></tr></thead><tbody><tr><td valign="top">Control</td><td valign="top">KnowledgeISSCCDIODR</td><td valign="top">NANANANA</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">1.06<xref ref-type="fn" rid="fn2" />.24<xref ref-type="fn" rid="fn2" />.10.04</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">1.32<xref ref-type="fn" rid="fn2" />.01.18.06</td></tr><tr><td valign="top">Standard Instruction</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">1.06<xref ref-type="fn" rid="fn2" />.24<xref ref-type="fn" rid="fn2" />.10.04</td><td valign="top">KnowledgeISSCCDIODR</td><td valign="top">NANANANA</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">.20<xref ref-type="fn" rid="fn2" />−.23<xref ref-type="fn" rid="fn2" />.07−.08</td></tr><tr><td valign="top">Enhanced Performance Feedback</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">1.32<xref ref-type="fn" rid="fn2" />.01.18.06</td><td valign="top">KnowledgeISSCCDIODR</td><td char="." valign="top">.20<xref ref-type="fn" rid="fn2" />−.23<xref ref-type="fn" rid="fn2" />.07.08</td><td valign="top">KnowledgeISSCCDIODR</td><td valign="top">NANANANA</td></tr><tr><td valign="top"><italic>Note</italic>. PF = performance feedback; ES = effect size; ISSC = Internalizing Symptoms Scale for Children; CDI = Children's Depression Inventory; ODR = office disciplinary referral.</td></tr><tr><td valign="top" /></tr></tbody></table> </ephtml> </p> <hd id="AN0059165621-17">Teacher Outcomes</hd> <p></p> <hd id="AN0059165621-18">Social validity of the Strong Kids program</hd> <p>Teachers' attitudes toward social and emotional learning and the Strong Kids program were evaluated by reviewing the responses on the Strong Kids <emph>Social Validity Survey</emph> that were completed by the teachers in the intervention groups. No formal qualitative analysis was performed due to having only two teachers respond to this survey; however, the responses were reviewed for themes and identification of areas of any difference. Teachers' attitudes were in general positive but mixed. Teachers were most closely aligned and positive regarding the importance of students having coping skills to deal with challenging life circumstances. Teachers' observations of students increase in knowledge were consistent with quantitative results of the same (i.e., students showed a large increase in knowledge of skills). The use of 15 min to prepare for lesson implementation was acceptable to the teachers. On the other hand, they were not satisfied with the amount of time it took to implement each lesson. This finding was surprising given there are other commercially available programs with content that requires more time to fully implement either with each lesson or the number of lessons to be implemented. Interestingly, both teachers found the amount of support they were given was acceptable; the teacher in the Enhanced Performance Feedback group received regular and interactive support whereas the No Consultation group teacher had no communication with the consultant. Given the small sample size (<emph>n</emph> = 2) to study this variable, reasons for this difference are speculative at best.</p> <hd id="AN0059165621-19">DISCUSSION</hd> <p>Overall, the students who participated in the Strong Kids program demonstrated significant gains in knowledge of the concepts and skills that are associated with healthy social and emotional behaviors. This finding is consistent with prior studies evaluating this issue ([<reflink idref="bib8" id="ref86">8</reflink>]; [<reflink idref="bib25" id="ref87">25</reflink>]; [<reflink idref="bib33" id="ref88">33</reflink>]). Effect size measurements indicated that not only did these students exhibit significantly higher knowledge scores than those students in the control group but also they did so to a large extent, and the students whose teacher received consultative support had more knowledge than the students whose teacher did not receive consultative support.</p> <p>Regarding the impact of the program on internalizing problems, the data do not support a significant reduction in symptoms over time and across groups. This finding is somewhat surprising given significant reductions in symptoms have been detected in prior studies ([<reflink idref="bib33" id="ref89">33</reflink>]). However, these previous studies included clinical samples with higher baseline rates of problem symptoms than the general education sample used in this study, which had relatively low problem symptoms to begin with. Although there were no significant effects detected, effect size calculations indicated that the students in the Standard Instruction group exhibited a small reduction in symptoms (compared with Control: ES =.24, over time: ES =.20) and the Control group experienced an increase in symptoms over time (ES =.20). Surprisingly, the Enhanced Performance Feedback group experienced a slight increase in symptoms, as measured by the CDI, compared with the Standard Instruction group. Given that the ISSC measure did not detect a similar finding, this result must be interpreted with caution. Low observed power could have contributed to difficulties in detecting both significant effects and trends. And certainly, the fact that the participants had a low base rate of symptoms from the start left few internalizing symptoms to reduce.</p> <p>The literature is promising regarding the effectiveness of using performance feedback to improve student outcomes (e.g., [<reflink idref="bib47" id="ref90">47</reflink>]) when it is applied to intervention efforts in a classroom setting. When considering SEL program implementation, it has been recommended that support during implementation should be available ([<reflink idref="bib48" id="ref91">48</reflink>]) to maximize student progress. Studying the effectiveness of using "support" with SEL programs is a relatively new area of study, the specifics not quite defined, and the extent of support that might be required is unknown. Most likely, there is not a "one-size-fits-all" approach to providing support, the parameters and intensity of which have yet to be determined. This study represents a pilot effort to study this issue and the results suggest uncertainty whether an enhanced performance feedback procedure is necessary to facilitate positive student outcomes and teacher satisfaction. With the exception of the students in the Enhanced Performance Feedback group who had more knowledge of concepts and skills at posttest, no other data suggested an advantage to students when their teacher received enhanced performance feedback. Given the minimum number of participants receiving enhanced performance feedback (1 teacher) and the number of consultants used (one) we also cannot rule out the possibility that the consultant had some kind of impact on increased knowledge.</p> <p>Of interest is the treatment integrity data suggesting that both teachers implemented the program lessons with a consistently high level of integrity. This finding is encouraging given the fact that the program was designed with a structured format in mind so that it may be more easily implemented. Both teachers expressed a high degree of motivation to implement the lessons "correctly," which probably contributed to high levels of integrity. Subsequently, although this attribute may certainly contribute to these findings, conclusions regarding whether enhanced performance feedback is necessary to promote treatment integrity may be premature given the high levels of integrity.</p> <p>Regarding teachers' satisfaction with the Strong Kids program, results were generally positive in terms of believing that these SEL skills are important, having observed their students gain knowledge of the material, and believing that the amount of time it took to prepare for each lesson was acceptable. Teachers did not like the amount of time it took to implement each lesson of the program. Interestingly, both teachers were satisfied with the amount of support they received—either no support during implementation or support with the enhanced performance feedback procedure. It is possible that the guidance already provided in the materials provides adequate support, but again, this is speculative given 1 participant in the Enhanced Performance Feedback group.</p> <hd id="AN0059165621-20">Limitations</hd> <p>Several limitations should be considered when interpreting these results. First, evaluating the effectiveness of a program that is geared toward prevention and early intervention with a population sample where the majority of individuals do not exhibit internalizing symptoms poses challenges for measuring change in the short term. On average, students reported symptoms within normal limits, indicating minimal concern for problems with depression and anxiety. Detecting change and reduction in symptoms is possible but most likely negligible when most students are functioning within normal limits. It is noteworthy that students' symptoms did not increase over time, a positive finding as children entering adolescence are known to experience the beginning of a steady trajectory of increased internalizing symptoms ([<reflink idref="bib31" id="ref92">31</reflink>]).</p> <p>Observed power for many of the analyses was relatively low, which can contribute to not detecting an effect when one truly exists. For example, observed power for detecting a significant interaction between Group and Time on internalizing symptoms using the CDI was.34 (<emph>p</emph> <.05), meaning that upon replication of this study, a false null hypothesis would not be rejected 66% of the time. Certainly, increasing the number of participants in the study would help mitigate this issue. Similarly, the number of teacher participants was minimal (<reflink idref="bib2" id="ref93">2</reflink>) so that detecting differences between the presence and absence of support (Enhanced Performance Feedback vs. Standard Instruction) is challenging. Of course use of randomization to treatment or control groups would have additionally strengthened the design. Although it is certainly preferable to randomize all participants to conditions, this study was limited by randomization only to intervention groups (Enhanced Performance Feedback vs. Standard Instruction) rather than randomly assigned teachers to an intervention group or a control group. This design issue was problematic, rendering exploratory results at best. As with many studies that evaluate effects over time using the same dependent measures, posttest scores can be inflated due to increased sensitivity that occurred at pretest. Finally, the demographic constitution of this student sample limits the generalizability of these results to similarly constructed schools and certainly poses challenges when hypothesizing the use of the program with more culturally diverse or different populations (e.g., urban, international schools).</p> <hd id="AN0059165621-21">Future Research</hd> <p>Goals of this study were to primarily evaluate a promising SEL program aimed at internalizing symptom reduction and second to explore whether an enhanced performance feedback procedure would contribute to the effectiveness of this program. Support for SEL programming has been recommended, but it has yet to be determined the type of support that would be appropriate and feasible. A performance feedback methodology was used as the framework for the enhanced performance feedback procedure in this study as performance feedback has been quite effective in producing positive student outcomes via teachers implementing some kind of instruction. Motivational interviewing techniques were used in conjunction with performance feedback in order to (a) apply an effective strategy to a situation where feedback is provided and resistance to change might occur, and (b) to introduce this technique as a viable option for use in the context of school-based consultation. Resistance is quite conceivable when teachers are given feedback that suggests they make changes to usual and customary classroom instructional or behavior management strategies. These changes, although potentially beneficial, may be unfamiliar or challenging and time consuming to implement. We believe future research should continue to examine this issue to determine the extent to which motivational interviewing, in conjunction with proven performance feedback strategies, may be effective and feasible for use with SEL learning programming and in the school setting in general. This collaborative methodology has the potential to enhance school-based consultation techniques in a manner that addresses resistance to change in a time in which demands on teachers are great, time is limited, and a respectful and collaborative consultation relationship is required. Of course, a comprehensive investigation of this process must include comparing and contrasting it with other known consultation methods (e.g., a problem-solving process) to determine a preferred method for use with an SEL program.</p> <p>This study did not examine the longitudinal effects of a program designed to not only ameliorate internalizing symptoms but also to act as a preventative agent against unknown but inevitable life stressors. Investing in such research will improve our understanding of (a) the extent to which the Strong Kids social and emotional learning program is able to produce such inoculative protection (i.e., youth will have learned and can apply skills that can help them cope), and (b) how durable these effects are over time.</p> <p>The concept of resiliency is not fully represented by measuring knowledge of skills and existing symptoms. Assessment of social and emotional strengths is an additional area that would provide a rich domain of information. This assessment would include youths' ability to apply these strengths and skills in times of stress. Particular areas of inquiry may include having a current support system, exhibiting good problem-solving skills, knowing how to soothe oneself when angry or upset, and being able to identify thinking errors. Similarly, multiple informants (e.g., parents, teachers) would add to not only the reliability and validity of this assessment but also to the comprehensive picture regarding youths' skill application in everyday environments.</p> <hd id="AN0059165621-22">Notes</hd> <ref id="AN0059165621-23"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref14" type="bt">1</bibl> <bibtext> <sups>a</sups>Denotes effect size greater than.19 and interpreted as a meaningful effect.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref26" type="bt">2</bibl> <bibtext> <sups>a</sups>Denotes effect size greater than.19 and interpreted as a meaningful effect.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref27" type="bt">3</bibl> <bibtext> The authors report that to the best of their knowledge neither they nor their affiliated institutions have financial or personal relationships or affiliations that could influence or bias the opinions, decisions, or work presented in this manuscript.</bibtext> </blist> </ref> <ref id="AN0059165621-24"> <title> REFERENCES </title> <blist> <bibtext> Ager, A. and O'May, F.2001. Issues in the definition and implementation of best practice for staff delivery of interventions for challenging behavior. Journal of Intellectual and Developmental Disability, 26: 243–256.</bibtext> </blist> <blist> <bibtext> Alvero, A.M., Bucklin, B.R. and Austin, J.2001. An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings. Journal of Organizational Behavior Management, 21: 3–29.</bibtext> </blist> <blist> <bibtext> Balcazar, F., Hopkins, B.L. and Suarez, Y.1985. A critical, objective review of performance feedback. Journal of Organizational Behavior Management, 7: 65–89.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref46" type="bt">4</bibl> <bibtext> Blom-Hoffman, J. and Rose, G.S.2007. Applying motivational interviewing to school-based consultation: A commentary on "Has consultation achieved its primary prevention potential?". Journal of Educational and Psychological Consultation, 17: 151–156.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref20" type="bt">5</bibl> <bibtext> Botvin, G.J., Baker, E., Dusenbury, L., Tortu, S. and Botvin, E.1990. Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3-year study. Journal of Consulting and Clinical Psychology, 58: 437–446.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref45" type="bt">6</bibl> <bibtext> Brown, J.M. and Miller, W.R.1993. Impact of motivational interviewing on participation and outcome of residential alcoholism treatment. Psychology of Addictive Behaviors, 7: 211–218.</bibtext> </blist> <blist> <bibl id="bib7" type="bt">7</bibl> <bibtext> CASEL. 2003. Safe and sound: An educational leaders' guide to evidence-based social and emotional learning (SEL) programsRetrieved from <ulink href="http://www.casel.org/downloads/Safe%20and%20Sound/1A%5fSafe%5f&%5fSound.pdf">http://www.casel.org/downloads/Safe%20and%20Sound/1A%5fSafe%5f&%5fSound.pdf</ulink></bibtext> </blist> <blist> <bibl id="bib8" idref="ref86" type="bt">8</bibl> <bibtext> Castro-Olivo, S.M.2006. The effects of a culturally-adapted social-emotional learning program on social-emotional and academic outcomes of Latino immigrant high school studentsEugene: University of Oregon.. (Unpublished doctoral dissertation)</bibtext> </blist> <blist> <bibl id="bib9" idref="ref85" type="bt">9</bibl> <bibtext> Cohen, J.1988. Statistical power analysis for the behavioral sciences, , 2nd ed.Hillsdale, NJ: Erlbaum..</bibtext> </blist> <blist> <bibtext> Coie, J.D., Miller-Jackson, S. and Bagwell, C.2000. "Prevention science.". In Handbook of developmental psychopathologyEdited by: Sameroff, A.J., Lewis, M. and Miller, S.M.93–108. New York, NY: Kluwer Academic/Plenum..</bibtext> </blist> <blist> <bibtext> Connell, D.B., Turner, R.R. and Mason, E.F.1985. Summary of findings of the school health education evaluation: Health promotion effectiveness, implementation, and costs. Journal of School Health, 55: 316–321.</bibtext> </blist> <blist> <bibtext> Cossairt, A., Hall, R.V. and Hopkins, B.L.1973. The effects of experimenter's instructions, feedback, and praise on teacher praise and student attending behavior. Journal of Applied Behavior Analysis, 6: 89–100.</bibtext> </blist> <blist> <bibtext> Cowen, E.L.1994. The enhancement of psychological wellness: Challenges and opportunities. American Journal of Community Psychology, 22: 149–179.</bibtext> </blist> <blist> <bibtext> Dishion, T.J. and Kavanagh, K.2003. Intervening in adolescent problem behavior: A family-centered approach.New York, NY: Guilford..</bibtext> </blist> <blist> <bibtext> Errecart, M.T., Walberg, H.J., Ross, J.G., Gold, R.S., Fiedler, J.L. and Kolbe, L.J.1991. Effectiveness of teenage health teaching modules. Journal of School Health, 61: 26–30.</bibtext> </blist> <blist> <bibtext> Espelage, D.L. and Swearer, S.M.2004. Bullying in American schools: A social-ecological perspective on prevention and interventionMahwah, NJ: Erlbaum..</bibtext> </blist> <blist> <bibtext> Fixsen, D.L., Naoom, S.F., Blasé, K.A., Friedman, R.M. and Wallace, F.2005. Implementation research: A synthesis of the literatureNational Implementation Research Network.. Retrieved from <ulink href="http://cfs.fmhi.usf.edu/%5fdocs/publications/NIRN%5fMonograph%5fFull.pdf">http://cfs.fmhi.usf.edu/%5fdocs/publications/NIRN%5fMonograph%5fFull.pdf</ulink></bibtext> </blist> <blist> <bibtext> Fuchs, D. and Fuchs, L.S.1996. Consultation as a technology and the politics of school reform. Remedial and Special Education, 17: 386–392.</bibtext> </blist> <blist> <bibtext> Gersten, R., Chard, D. and Baker, S.2000. Factors enhancing sustained use of research-based instructional practices. Journal of Learning Disabilities, 33: 445–457.</bibtext> </blist> <blist> <bibtext> Gresham, F.M.1989. Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18: 37–50.</bibtext> </blist> <blist> <bibtext> Gresham, F.M. and Lopez, M.E.1996. Social validation: A unifying concept for school-based consultation research and practice. School Psychology Quarterly, 11: 204–227.</bibtext> </blist> <blist> <bibtext> Harchick, A.E., Sherman, J.A., Sheldon, J.B. and Strouse, M.C.1992. Ongoing consultation as a method of improving performance of staff members in a group home. Journal of Applied Behavior Analysis, 25: 599–610.</bibtext> </blist> <blist> <bibtext> Henggeler, S.W., Melton, G.B., Brondino, M.J., Scherer, D.G. and Hanley, J.H.1997. Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65: 821–833.</bibtext> </blist> <blist> <bibtext> Heron, T.E. and Harris, K.C.2001. The educational consultant: Helping professionals, parents, and students in inclusive classrooms.Austin, TX: Pro-Ed..</bibtext> </blist> <blist> <bibtext> Isava, D.M.2006. An investigation of the impact of the Strong Kids curriculum on problem symptoms and knowledge gains among adolescents in a residential treatment centerEugene: University of Oregon.. (Unpublished doctoral dissertation)</bibtext> </blist> <blist> <bibtext> Joyce, B. and Showers, B.1988. Student achievement through staff developmentNew York, NY: Longman..</bibtext> </blist> <blist> <bibtext> Kauffman, J.M. and Landrum, T.J.2009. Characteristics of emotional and behavioral disorders of children and youth, , 9th ed.Upper Saddle River, NJ: Merrill Prentice-Hall..</bibtext> </blist> <blist> <bibtext> Keppel, G. and Zedeck, S.1989. Data analysis for research designs: Analysis of variance and multiple regression/correlation approaches.New York, NY: Freeman..</bibtext> </blist> <blist> <bibtext> Kovacs, M.1992. The Children's Depression Inventory.San Antonio, TX: Pearson Assessments..</bibtext> </blist> <blist> <bibtext> Kratochwill, T.R., Elliott, S.N. and Callan-Stoiber, K.2002. "Best practices in school-based problem-solving consultation.". In Best practices in school psychology IVEdited by: Thomas, A. and Grimes, J.503–516. Bethesda, MD: National Association of School Psychologists..</bibtext> </blist> <blist> <bibtext> Lewinsohn, P.M., Clarke, G.N., Seeley, J.R. and Rohde, P.1994. Major depression in community adolescents: Age of onset, episode duration, and time to recurrence. Journal of the American Academy of Child and Adolescent Psychiatry, 33: 809–818.</bibtext> </blist> <blist> <bibtext> Lipsey, M.W. and Wilson, D.B.2001. Practical meta-analysis.Thousand Oaks, CA: Sage..</bibtext> </blist> <blist> <bibtext> Merrell, K.W.2008. Helping students overcome depression and anxiety: A practical guide, , 2nd ed.New York, NY: Guilford..</bibtext> </blist> <blist> <bibtext> Merrell, K.W., Blade, R.L., Lund, J. and Kempf, K.K. G.2003. Convergent and discrimination construct validity of the Internalizing Symptoms Scale for Children with the BASC-SRP-C. Psychology in the Schools, 40: 139–144.</bibtext> </blist> <blist> <bibtext> Merrell, K.W. and Buchanan, R.S.2006. Intervention selection in school-based practice: Using public health models to enhance systems capacity of schools. School Psychology Review, 35: 167–180.</bibtext> </blist> <blist> <bibtext> Merrell, K.W., Carrizales, D., Feuerborn, L., Gueldner, B.A. and Tran, O.K.2007. Strong kids: A social and emotional learning programBaltimore, MD: Brookes..</bibtext> </blist> <blist> <bibtext> Merrell, K.W., Juskelis, M.P., Buchanan, R. and Tran, O.K.2008. Social and emotional learning in the classroom: Evaluation of Strong Kids and Strong Teens on students' social-emotional knowledge and symptoms. Journal of Applied School Psychology, 24: 209–224.</bibtext> </blist> <blist> <bibtext> Merrell, K.W., McClun, L.A., Kempf, K.K. G. and Lund, J.2002. Using self-report assessment to identify children with internalizing problems: Validity of the Internalizing Symptoms Scale for Children. Journal of Psychoeducational Assessment, 20: 223–239.</bibtext> </blist> <blist> <bibtext> Merrell, K.W. and Walters, A.S.1998. Internalizing Symptoms Scale for Children.Austin, TX: Pro-Ed..</bibtext> </blist> <blist> <bibtext> Miller, W.R. and Rollnick, S.2002. Motivational interviewing: Preparing people for change.New York, NY: Guilford..</bibtext> </blist> <blist> <bibtext> Mortenson, B.P. and Witt, J.C.1998. The use of weekly performance feedback to increase teacher implementation of a prereferral academic intervention. School Psychology Review, 27: 613–627.</bibtext> </blist> <blist> <bibtext> Nakayama, N.J.2008. An investigation of the impact of the Strong Kids curriculum on social-emotional knowledge and symptoms of elementary aged students in a self-contained special education settingEugene: University of Oregon.. (Unpublished doctoral dissertation)</bibtext> </blist> <blist> <bibtext> Noell, G.H., Duhon, G.J., Gatti, S.L. and Connell, J.E.2002. Consultation, follow-up, and implementation of behavior management interventions in general education. School Psychology Review, 31: 217–234.</bibtext> </blist> <blist> <bibtext> Noell, G.H., Gresham, F.M. and Gansle, K.A.2002. Does treatment integrity matter? A preliminary investigation of instructional implementation and mathematics performance. Journal of Behavioral Education, 11: 51–67.</bibtext> </blist> <blist> <bibtext> Noell, G.H., Witt, J.C., Gilbertson, D.N., Ranier, D.D. and Freeland, J.T.1997. Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12: 77–88.</bibtext> </blist> <blist> <bibtext> Noell, G.H., Witt, J.C., LaFleur, L.H., Mortenson, B.P., Ranier, D.D. and LeVelle, J.2000. Increasing intervention implementation in general education following consultation: A comparison of two follow-strategies. Journal of Applied Behavior Analysis, 33: 271–284.</bibtext> </blist> <blist> <bibtext> Noell, G.H., Witt, J.C., Slider, N.J., Connell, J.E., Gatti, S.L., Williams..., K.L. and Resetar, J.L.2005. Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34: 87–106.</bibtext> </blist> <blist> <bibtext> Payton, J.W., Wardlaw, D.M., Graczyk, P.A., Bloodworth, M.R., Tompsett, C.J. and Weissberg, R.P.2000. Social and emotional learning: A framework for promoting mental health and reducing risk behaviors in children and youth. Journal of School Health, 70: 179–185.</bibtext> </blist> <blist> <bibtext> Reinke, W.M., Lewis-Palmer, T. and Merrell, K.W.2008. The classroom check-up: A classwide teacher consultation model for increasing praise and decreasing disruptive behavior. School Psychology Review, 37: 315–332.</bibtext> </blist> <blist> <bibtext> Ross, J.G., Luepker, R.V., Nelson, G.D., Saavedra, P. and Hubbard, B.M.1991. Teenage health teaching modules: Impact of teacher training on implementation and student outcomes. Journal of School Health, 61: 31–34.</bibtext> </blist> <blist> <bibtext> Schwartz, I.S. and Baer, D.M.1991. Social validity assessments: Is current practice state of the art?. Applied Behavior Analysis, 24: 189–204.</bibtext> </blist> <blist> <bibtext> Seeley, J.R., Rohde, P., Lewinsohn, P.M. and Clarke, G.N.2002. "Depression in youth: Epidemiology, identification, and intervention.". In Interventions for academic and behavior problems II: Preventative and remedial approachesEdited by: Shinn, M.R., Walker, H.M. and Stoner, G.885–911. Bethesda, MD: National Association of School Psychologists..</bibtext> </blist> <blist> <bibtext> Sheridan, S.M. and Cowan, R.J.2004. "Consultation with school personnel.". In Handbook of pediatric psychologists in school settingsEdited by: Brown, R.T.599–616. Mahwah, NJ: Erlbaum..</bibtext> </blist> <blist> <bibtext> Smith, D.W., McCormick, L.K., Steckler, A.B. and McLeroy, K.R.1993. Teachers' use of health curricula: Implementation of growing healthy, projectSMART, and the teenage health teaching modules. Journal of School Health, 63: 349–354.</bibtext> </blist> <blist> <bibtext> Stokes, T.F. and Baer, D.M.1977. An implicit technology of generalization. Journal of Applied Behavior Analysis, 10: 349–367.</bibtext> </blist> <blist> <bibtext> Telzrow, C.F. and Beebe, J.J.2002. "Best practices in facilitating intervention adherence and integrity.". In Best practices in school psychology IVEdited by: Thomas, A. and Grimes, J.503–516. Bethesda, MD: National Association of School Psychologists..</bibtext> </blist> <blist> <bibtext> Tran, O.K.2007. Promoting social and emotional learning in schools: An investigation of massed versus distributed practice schedules and social validity of the Strong Kids program in late elementary aged studentsEugene: University of Oregon.. (Unpublished doctoral dissertation)</bibtext> </blist> <blist> <bibtext> Wang, M.C., Haertel, G.D. and Walberg, H.J.1997. "Fostering educational resilience in inner-city schools.". In Children and youth: Interdisciplinary perspectivesEdited by: Walber, H.J., Reyes, O. and Weissberg, R.P.46–65. Thousand Oaks, CA: Sage..</bibtext> </blist> <blist> <bibtext> Witt, J.C., Noell, G., LaFleur, L. and Mortenson, B.P.1997. Teacher use of intervention in general education setting: Measurement and analysis of the independent variable. Journal of Applied Behavior Analysis, 30: 693–696.</bibtext> </blist> <blist> <bibtext> World Health Organization. 2004. Prevention of mental disorders: Effective interventions and policy options: A summary report.Geneva, , Switzerland: World Health Organization Department of Mental Health and Substance Abuse..</bibtext> </blist> <blist> <bibtext> Zins, J.E., Bloodworth, M.R., Weissberg, R.P. and Walberg, H.J.2004. "The scientific base linking social and emotional learning to school success.". In Building academic success on social and emotional learning: What does the research say?Edited by: Zins, J.E., Weissberg, R.P., Wang, M.C. and Walberg, H.J.New York, NY: Teachers College Press..</bibtext> </blist> </ref> <aug> <p>By Barbara Gueldner and Kenneth Merrell</p> <p>Reported by Author; Author</p> <p></p> <p>Barbara Gueldner, PhD, is a Post-Doctoral Fellow in the Stress and Anxiety Clinic and Child Health Clinic at The Children's Hospital in Aurora, Colorado. She has over 10 years experience as a school psychologist, has worked with children and their families in community and hospital-based clinics, and has clinical and research interests in the areas of social-emotional learning and prevention and early intervention of mental health in pediatric populations. She received a Masters of Science in Education in School Psychology from the University of Wisconsin-Whitewater and a PhD in School Psychology from the University of Oregon.</p> <p>Kenneth Merrell, PhD, is a Professor of School Psychology at the University of Oregon, where he serves as head of the Department of Special Education and Clinical Sciences. He is a Fellow in the American Psychological Association's Division of School Psychology (Division 16) and Society for Clinical Child and Adolescent Psychology (Division 53). He received his PhD in School Psychology from the University of Oregon. Dr. Merrell's research and clinical interests are on social-emotional learning and social-emotional assessment practices in schools.</p> </aug> <nolink nlid="nl1" bibid="bib16" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib27" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib33" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib52" firstref="ref4"></nolink> <nolink nlid="nl5" bibid="bib58" firstref="ref5"></nolink> <nolink nlid="nl6" bibid="bib61" firstref="ref6"></nolink> <nolink nlid="nl7" bibid="bib36" firstref="ref7"></nolink> <nolink nlid="nl8" bibid="bib57" firstref="ref9"></nolink> <nolink nlid="nl9" bibid="bib48" firstref="ref10"></nolink> <nolink nlid="nl10" bibid="bib26" firstref="ref11"></nolink> <nolink nlid="nl11" bibid="bib55" firstref="ref12"></nolink> <nolink nlid="nl12" bibid="bib59" firstref="ref13"></nolink> <nolink nlid="nl13" bibid="bib17" firstref="ref15"></nolink> <nolink nlid="nl14" bibid="bib47" firstref="ref17"></nolink> <nolink nlid="nl15" bibid="bib53" firstref="ref18"></nolink> <nolink nlid="nl16" bibid="bib24" firstref="ref19"></nolink> <nolink nlid="nl17" bibid="bib11" firstref="ref21"></nolink> <nolink nlid="nl18" bibid="bib15" firstref="ref22"></nolink> <nolink nlid="nl19" bibid="bib50" firstref="ref23"></nolink> <nolink nlid="nl20" bibid="bib54" firstref="ref24"></nolink> <nolink nlid="nl21" bibid="bib41" firstref="ref28"></nolink> <nolink nlid="nl22" bibid="bib43" firstref="ref29"></nolink> <nolink nlid="nl23" bibid="bib45" firstref="ref30"></nolink> <nolink nlid="nl24" bibid="bib46" firstref="ref31"></nolink> <nolink nlid="nl25" bibid="bib18" firstref="ref33"></nolink> <nolink nlid="nl26" bibid="bib19" firstref="ref34"></nolink> <nolink nlid="nl27" bibid="bib22" firstref="ref40"></nolink> <nolink nlid="nl28" bibid="bib30" firstref="ref43"></nolink> <nolink nlid="nl29" bibid="bib56" firstref="ref50"></nolink> <nolink nlid="nl30" bibid="bib51" firstref="ref51"></nolink> <nolink nlid="nl31" bibid="bib20" firstref="ref52"></nolink> <nolink nlid="nl32" bibid="bib23" firstref="ref54"></nolink> <nolink nlid="nl33" bibid="bib42" firstref="ref63"></nolink> <nolink nlid="nl34" bibid="bib34" firstref="ref66"></nolink> <nolink nlid="nl35" bibid="bib38" firstref="ref67"></nolink> <nolink nlid="nl36" bibid="bib21" firstref="ref68"></nolink> <nolink nlid="nl37" bibid="bib28" firstref="ref69"></nolink> <nolink nlid="nl38" bibid="bib32" firstref="ref83"></nolink> <nolink nlid="nl39" bibid="bib35" firstref="ref84"></nolink> <nolink nlid="nl40" bibid="bib25" firstref="ref87"></nolink> <nolink nlid="nl41" bibid="bib31" firstref="ref92"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ917352
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluation of a Social-Emotional Learning Program in Conjunction with the Exploratory Application of Performance Feedback Incorporating Motivational Interviewing Techniques
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Gueldner%2C+Barbara%22">Gueldner, Barbara</searchLink><br /><searchLink fieldCode="AR" term="%22Merrell%2C+Kenneth%22">Merrell, Kenneth</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Educational+%26+Psychological+Consultation%22"><i>Journal of Educational & Psychological Consultation</i></searchLink>. 2011 21(1):1-27.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: PhysDesc
  Label: Physical Description
  Group: PhysDesc
  Data: PDF
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 27
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2011
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Evaluative
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Early+Intervention%22">Early Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Interviews%22">Interviews</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Prevention%22">Prevention</searchLink><br /><searchLink fieldCode="DE" term="%22Behavior+Problems%22">Behavior Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Child+Development%22">Child Development</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Descriptions%22">Program Descriptions</searchLink><br /><searchLink fieldCode="DE" term="%22Resilience+%28Psychology%29%22">Resilience (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Symptoms+%28Individual+Disorders%29%22">Symptoms (Individual Disorders)</searchLink><br /><searchLink fieldCode="DE" term="%22Motivation+Techniques%22">Motivation Techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Social+Development%22">Social Development</searchLink><br /><searchLink fieldCode="DE" term="%22Emotional+Development%22">Emotional Development</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1080/10474412.2010.522876
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1047-4412
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The prevention of and early intervention with internalizing problems are vital to children's overall health and development. Schools are strongly considered a venue where these services can be provided. The Strong Kids program was developed to increase youths' resiliency. This study examined the efficacy of Strong Kids with middle school students in terms of knowledge of healthy social-emotional behaviors, reduction in internalizing symptoms, and the exploratory use of performance feedback with features of motivational interviewing. Students who participated in the program significantly increased their knowledge. Negative symptoms did not change. Teachers implemented the program with strong fidelity. It is unclear at this time the extent to which it is necessary to use performance feedback with features of motivational interviewing to improve the implementation of highly structured social-emotional learning programs. (Contains 6 tables.)
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 61
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2011
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ917352
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ917352
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/10474412.2010.522876
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 27
        StartPage: 1
    Subjects:
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: Early Intervention
        Type: general
      – SubjectFull: Interviews
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
      – SubjectFull: Prevention
        Type: general
      – SubjectFull: Behavior Problems
        Type: general
      – SubjectFull: Child Development
        Type: general
      – SubjectFull: Program Descriptions
        Type: general
      – SubjectFull: Resilience (Psychology)
        Type: general
      – SubjectFull: Symptoms (Individual Disorders)
        Type: general
      – SubjectFull: Motivation Techniques
        Type: general
      – SubjectFull: Social Development
        Type: general
      – SubjectFull: Emotional Development
        Type: general
    Titles:
      – TitleFull: Evaluation of a Social-Emotional Learning Program in Conjunction with the Exploratory Application of Performance Feedback Incorporating Motivational Interviewing Techniques
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Gueldner, Barbara
      – PersonEntity:
          Name:
            NameFull: Merrell, Kenneth
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2011
          Identifiers:
            – Type: issn-print
              Value: 1047-4412
          Numbering:
            – Type: volume
              Value: 21
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Journal of Educational & Psychological Consultation
              Type: main
ResultId 1