View in EDS HTML Full Text PDF Full Text

Reliability and Utility of the Behaviour Support Plan Quality Evaluation Tool (BSP-QEII) for Auditing and Quality Development in Services for Adults with Intellectual Disability and Challenging Behaviour

Saved in:
Bibliographic Details
Title: Reliability and Utility of the Behaviour Support Plan Quality Evaluation Tool (BSP-QEII) for Auditing and Quality Development in Services for Adults with Intellectual Disability and Challenging Behaviour
Language: English
Authors: McVilly, K., Webber, L., Paris, M., Sharp, G.
Source: Journal of Intellectual Disability Research. Aug 2013 57(8):716-727.
Availability: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA/
Peer Reviewed: Y
Page Count: 12
Publication Date: 2013
Document Type: Journal Articles
Reports - Research
Descriptors: Foreign Countries, Behavior Problems, Behavior Modification, Adults, Mental Retardation, Day Programs, Test Validity, Test Reliability, Planning, Quality Control, Program Effectiveness, Intervention, Interrater Reliability, Teamwork, Predictor Variables, Environmental Influences, Reinforcement, Teaching Methods, Responses, Goal Orientation, Interpersonal Communication, Audits (Verification)
Geographic Terms: Australia
DOI: 10.1111/j.1365-2788.2012.01603.x
ISSN: 0964-2633
Abstract: Background: Having an objective means of evaluating the quality of behaviour support plans (BSPs) could assist service providers and statutory authorities to monitor and improve the quality of support provided to people with intellectual disability (ID) who exhibit challenging behaviour. The Behaviour Support Plan Quality Evaluation Guide II (BSP-QEII) was developed to monitor and assess BSPs prepared by teachers to support children with disability in the school system. This study investigated the application of the BSP-QEII to the assessment of BSPs for adults with ID in community support services. Method: The inter-rater reliability of the BSP-QEII was assessed. The utility of the BPS-QEII was then investigated with reference to a time series study of matched pairs of BSPs, developed for the same clients over a period of approximately 3 years. Differences in plan quality measured across a number of service and systemic variables were also investigated. Results: The BSP-QEII was found to have good inter-rater reliability and good utility for audit purposes. It was able to discriminate changes in plan quality over time. Differences in plan quality were also evident across different service types, where specialist staff had or had not been involved, and in some instances where a statutory format for the plan had or had not been used. There were no differences between plans developed by government and community sector agencies, nor were there any regional differences across the jurisdiction. Conclusions: The BSP-QEII could usefully be adopted as an audit tool for measuring the quality of BSPs for adults with ID. In addition to being used for research and administrative auditing, the principles underpinning the BSP-QEII could also be useful to guide policy and educational activities for staff in community based services for adults with ID. (Contains 4 tables and 1 figure.)
Abstractor: As Provided
Number of References: 29
Entry Date: 2014
Accession Number: EJ1014798
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGczdZjIWTtSWBOpsrLolBGAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDOOsGq10AGqrs8mlsAIBEICBmnWyGKb5eDpycmEQHmrYjXqZgRdZ_RPdKQ1Q_5L6UmhbC5vQB47ZeGgQNdgOw2uDduN9X37-dTZzZs6VOFr58KrbOTjV7hlXKh7XqYMMpTWOwlpr_1wVv3axRDO3Np5iXNbbdYhxcbee3JmE31V_ArqMPoiKQPcGqy-PR8OSvid71Efa6F2VNKbkFUVeJAFgYzWNTq9EdCwznXU=
Text:
  Availability: 1
  Value: <anid>AN0088263558;eul01aug.13;2018Jul09.14:14;v2.2.500</anid> <title id="AN0088263558-1">Reliability and utility of the Behaviour Support Plan Quality Evaluation tool (BSP-QEII) for auditing and quality development in services for adults with intellectual disability and challenging behaviour. </title> <p>Background  Having an objective means of evaluating the quality of behaviour support plans (BSPs) could assist service providers and statutory authorities to monitor and improve the quality of support provided to people with intellectual disability (ID) who exhibit challenging behaviour. The Behaviour Support Plan Quality Evaluation Guide II (BSP‐QEII) was developed to monitor and assess BSPs prepared by teachers to support children with disability in the school system. This study investigated the application of the BSP‐QEII to the assessment of BSPs for adults with ID in community support services. Method  The inter‐rater reliability of the BSP‐QEII was assessed. The utility of the BPS‐QEII was then investigated with reference to a time series study of matched pairs of BSPs, developed for the same clients over a period of approximately 3 years. Differences in plan quality measured across a number of service and systemic variables were also investigated. Results  The BSP‐QEII was found to have good inter‐rater reliability and good utility for audit purposes. It was able to discriminate changes in plan quality over time. Differences in plan quality were also evident across different service types, where specialist staff had or had not been involved, and in some instances where a statutory format for the plan had or had not been used. There were no differences between plans developed by government and community sector agencies, nor were there any regional differences across the jurisdiction. Conclusions  The BSP‐QEII could usefully be adopted as an audit tool for measuring the quality of BSPs for adults with ID. In addition to being used for research and administrative auditing, the principles underpinning the BSP‐QEII could also be useful to guide policy and educational activities for staff in community based services for adults with ID.</p> <p>behaviour support plan; challenging behaviour; reliability; assessment</p> <p>It is estimated that between 7% and 42% of people with intellectual disability (ID) exhibit challenging behaviour (Allen & Hill‐Tout 1998; Emerson et al. 2001; Crocker et al. 2006; Cooper et al. 2009; Emerson & Einfeld 2011). The large variation in these figures could be explained, in part at least, by methodological variations in the studies. These include differing definitions which have been adopted as to what constitutes challenging behaviour, and the heterogeneity of places in which these data have been gathered. A recent total population study of the use of restrictive interventions (i.e. chemical and mechanical restraint, and seclusion) in Victoria, Australia indicated that at least 9% of people receiving a support service exhibited behaviours of sufficient severity that a reportable intervention had been used (Webber et al. 2010). This is considered a conservative population estimate, as these figures do not include physical restraint which until recently was not, on its own, a reportable intervention.</p> <p>What can confidently be surmised from the literature is that challenging behaviours persist over time, and as such represent a serious risk to the health and wellbeing of people with ID and those who provide their support (Emerson & Einfeld 2011). Furthermore, the view that one can ‘fix’ challenging behaviour by means of short term intervention is clearly not in line with research evidence (cf. Bird & Luiselli 2000; McClean et al. 2007). Therefore, long‐term evidence‐based interventions that include pro‐active strategies towards enhancing environments, building individual strengths and increasing life skills are vital (Ball et al. 2004; Carr & Horner 2007; Australian Psychological Society, 2011).</p> <p>To achieve these ends, and because of the complexity of both people's behavioural support needs and the service system typically providing their support, a planned and well‐structured approach is required (Feldman et al. 2004). This is usually facilitated by means of a document referred to as a ‘behaviour management’ or ‘behaviour support plan’ (BSP). That is, a plan developed for a person with a disability which specifies a range of strategies to be used when supporting the person and, which in addition to managing the person's behaviour, includes proactive strategies to build on the person's strengths and increase their skills and quality of life (Horner et al. 2000). Furthermore, such plans are typically informed by and build upon the principles and practices of Positive Behaviour Support (Carr et al. 1999). These plans provide both clinical guidance and act as a legal safeguard to protect the human and civil rights and interests of both people with disability and those providing support services.</p> <p>In Victoria, Australia, the Office of the Senior Practitioner (OSP) has been created to protect the human and civil rights of people with disability who exhibit challenging behaviours and who, as a consequence, are at risk of being subject to restrictive practices (Disability Act [Victoria], 2006, Division 5). As part of its role, the OSP is mandated to review all BSPs for people subject to restrictive interventions, and provide advice and education concerning behaviour support and intervention practices.</p> <p>Having a strong legislative framework and standardised planning formats is one means by which the quality of BSPs might be enhanced and, as a consequence quality of life outcomes for persons with ID (Blood & Neel 2007; Cook et al. 2012). Furthermore, a strong focus on the quality of BSPs could also influence quality outcomes for people with disability (Webber et al. 2011a). However, to date it has proven difficult to identify an objective measure that is valid, reliable, and easily implemented to assess the quality of BSPs submitted to the Senior Practitioner for review.</p> <p>Phillips et al. (2010) are among the first researchers to attempt to assess the quality of BSPs pre and post the new legislation in Victoria. These authors provide valuable baseline data pertaining to the quality of BSP's and highlight important areas for practice improvement. However, their study relied on an audit tool that the authors themselves developed, for which only limited details are available concerning the validity of its items and their reliability. Furthermore, the results reported are based on a limited sample sourced from a single community service organisation.</p> <p>The Behaviour Support Plan Quality Evaluation Guide II (BSP‐QEII) is a standardised research‐based scoring instrument designed to objectively evaluate and rate the quality of content in BSPs (Cook et al. 2006; Browning‐Wright et al. 2007). The BSP‐QEII encompasses six key concepts of BSP development, asserted by the authors to reflect contemporary good practice in behaviour support (Browning‐Wright et al. 2003):</p> <p>All behaviours, including problem behaviour, serve a purpose, allowing the student to get a need met (i.e. behaviour serves a function);</p> <p>Behaviour is related to the context/environment in which it occurs;</p> <p>There are two strands to a comprehensive BSP. Changing behaviour requires addressing the environmental features (removing the need for use of problem behaviour to get needs met), and teaching a functionally equivalent behaviour that student can use to get that same need met in an acceptable way;</p> <p>New behaviour must be reinforced to result in maintenance over time;</p> <p>Implementers need to know how to handle problem behaviour if it occurs again; and</p> <p>Communication needs to be between all important stakeholders, frequently enough to result in the continuous teaming necessary to achieve success.</p> <p>These six concepts in turn have informed the development of 12 practice standards which comprise the BSP‐QEII criteria. The 12 practice standards are presented later in the current study.</p> <p>The BSP‐QEII has the potential to be used as a tool to assess the quality of BSPs, as part of a jurisdiction‐wide monitoring system designed to ensure service providers deliver supports and interventions that are consistent with evidence‐based practice. To date the BSP‐QEII has been applied successfully to the assessment of BSPs developed in US jurisdictions and applied to the support of children with special education needs in school settings (Cook et al. 2006, 2012; Kraemer et al. 2008). Past research has reported strong inter‐rater reliability (IRR) among the items of the BSP‐QEII, ranging from 0.78 (Kraemer et al. 2008) to 0.84 (Cook et al. 2012). The extent to which these findings extend to the assessment of BSPs developed for adults with ID utilising community services remains unknown.</p> <p>The current study was conducted as part of a larger research programme investigating the application of the BSP‐QEII to the assessment of BSPs prepared for adults with ID, supported by a range of post‐school community‐based disability services. The larger study included piloting the utility of the BSP‐QEII (Webber et al. 2011b), and an investigation into the content validity of the BSP‐QEII, with reference to an expert group of behaviour support practitioners working in adult services (McVilly et al. 2012). This component of the study investigated the IRR of the BSP‐QEII, for use as an audit tool to assess the quality of BSPs for adults with ID in supported accommodation and other community‐based services. The study also investigated the utility of the BSP‐QEII for auditing the quality of plans submitted as a statutory requirement, and for monitoring changes in plan quality over time. The extent to which the BSP‐QEII could identify variations in quality across different types of service and forms of service provision was also investigated.</p> <hd id="AN0088263558-2">Method</hd> <hd id="AN0088263558-3">Ethics</hd> <p>Ethics approval for this study was granted by the Deakin University Human Research Ethics Committee. Access to data was authorised by the Department of Human Services, Office of the Senior Practitioner, under the provisions of the Disability Act [Victoria], 2006 (sections 24 & 26). In addition, the researchers were subject to a confidentiality agreement with the Department of Human Services, with respect to any information that might have identified an individual client or service provider.</p> <hd id="AN0088263558-4">Procedure</hd> <p>The study was conducted in two parts. The first investigated the IRR of the BSP‐QEII, when used to assess BSPs for adults with ID. Secondly, the utility of the tool was investigated with its application to the assessment of a sample of BSPs submitted, as a statutory requirement, to a government agency for evaluation.</p> <p>Prior to commencing the investigation, a research assistant (a student undertaking graduate studies in psychology, and without any previous training relating to services for people with ID) undertook training in the use of the BSP‐QEII. The training commenced with their participation in a 2‐day workshop (approximately 12 h in total), in which they were accompanied by experienced behaviour support practitioners. The workshop involved a detailed presentation of the 12 key principles of the BSP‐QEII and the assessment of four mock BSPs. Three of these plans which were open to group discussion so as to provide clarity and increased guidance around navigating the assessment criteria. The fourth plan was used in the workshop as an individual exercise, to consolidate learning and provide the workshop leaders with an opportunity to engage the attendees in some individual mentorship. A handout on ‘Improving Behaviour Support Planning’ was given to those attending the workshop to assist in scoring the BSPs using the BSP‐QEII.</p> <p>Following training the research assistant then participated in an induction phase, mentored by an experienced project officer who had previously been trained to use the BSP‐QEII and had assessed approximately 350 BSPs using the BSP‐QEII. During the induction phase, the IRR of the BSP‐QEII was assessed on two occasions (first following assessment of 10 plans, and then following assessment of a further seven plans). At the completion of each stage of induction, the researchers discussed the BSPs among members of the team. IRR is the psychometric measure of choice to determine reliability of an assessment tool, as it quantifies the extent of agreement between different raters (Gwet 2010). IRR is generally considered acceptable at 80% agreement (Anastasi & Urbina 1997), For the purposes of the current study, induction was judged to be complete when the research assistant reached 80% agreement with the experienced project officer, with kappa (a measure of agreement on categorical items, which takes into account agreement occurring by chance alone) set at >0.4 (i.e. at least moderate agreement) (Landis & Koch 1977).</p> <p>After the induction phase was complete, the researchers undertook an assessment of the BSP‐QEII's utility by means of an assessment of a random sample of BSPs which had been submitted, as a statutory requirement, to the OSP. In this phase of the study a time series design was employed to also investigate the quality of BSPs submitted to the Senior Practitioner in 2006/2007 (Time One), compared to those for the same clients in 2009/2010 (Time Two). The researchers alternated between rating plans developed at Time One and Time Two. In all instances, data were first entered into Microsoft Excel and later transferred to the Statistical Package for the Social Sciences (spss), Version 17 for analysis. Total BSP‐QEII scores were compared using Wilcoxon Signed Rank Test between plans developed at Time One and those developed at Time Two, with P set at <0.05. Data were also compared across several organisational characteristics, hypothesised to influence the quality of BSPs.</p> <hd id="AN0088263558-5">Materials</hd> <p>The ratings were conducted according to the standardised BSP‐QEII criteria (Browning‐Wright et al. 2003). The BSP‐QEII scoring guide is built upon 12 good practice principles drawn from the research literature. Each of the 12 components is rated on a Likert scale (0 = no evidence, 1 = partial evidence or 2 = clear evidence) with each component having a unique scoring criterion to produce a score range of 0 to 24. The BSP‐QEII total score obtained is then classified into one of four categories, based on recommendations of its developers: ‘Superior’ (22–24 points), ‘Good’ (17–21 points), ‘Underdeveloped’ (13–16 points) or ‘Weak’ (12 points or less).</p> <p>The plans used for the ratings were randomly selected from those submitted to the OSP in 2006/2007 (Time One), representing the first batch of plans to be submitted under the new legislation. However, the inclusion criteria also required that a matched pair (i.e. a plan for the same client prepared by the same agency) be available for assessment which had been submitted in the reporting year 2009/2010 (Time Two). Subsequently, there were 174 plans assessed, equating to 87 matched pairs. These plans represented approximately 5% of those submitted at either Time One of Time Two. Profiles relating to the plans under assessment are given in Table 1.</p> <p>1 Origin of, contributors to, and format for behaviour support plans</p> <p> <ephtml> <table><tr><th>Characteristic of behaviour support plan</th><th align="center">Number of behaviour support plans</th></tr><tr><td>Service sector:</td><td>Government</td><td>Community sector</td></tr><tr><td>42 pairs (84)</td><td>45 pairs (90)</td></tr><tr><td>Service type:</td><td>SSA</td><td>Day support</td><td>Respite</td><td>CC</td></tr><tr><td>120</td><td>23</td><td>23</td><td>8</td></tr><tr><td>Administrative region:</td><td>Metro 1</td><td>49</td><td /></tr><tr><td>Metro 2</td><td>27</td><td /></tr><tr><td>Metro 3</td><td>39</td><td /></tr><tr><td>Rural 1</td><td>22</td><td /></tr><tr><td>Rural 2</td><td>12</td><td /></tr><tr><td>Rural 3</td><td>6</td><td /></tr><tr><td>Rural 4</td><td>11</td><td /></tr><tr><td>Rural 5</td><td>8</td><td /></tr><tr><td>Behaviour support team involvement*:</td><td>None</td><td>Consulted†</td><td>Author</td></tr><tr><td>134</td><td>31</td><td>9</td></tr><tr><td>Format:</td><td>OSP</td><td>OSP‐modified</td><td>Agency's own</td></tr><tr><td>73</td><td>18</td><td>83</td></tr></table> </ephtml> </p> <p>1 * Behaviour Intervention Support Teams (BIST).</p> <ulist> <item>2 For a behaviour support plan to meet the criteria of ‘Consulted’ requires face to face meetings and/or reports prepared by BIST practitioners.</item> <item>3 CC, Congregate Care (older institutional facilities); OSP, Office of the Senior Practitioner; SSA, Shared Supported Accommodation (shared houses in the community).</item> </ulist> <hd id="AN0088263558-6">Results</hd> <hd id="AN0088263558-7">Inter‐rater reliability</hd> <p>The BSP‐QEII total scores were calculated on two occasions during the research assistant's induction. Preliminary analysis at Stage 1 revealed that assumptions of normality, linearity and homoscedasticity for the distribution of scores were met for Rater One (K‐S = 0.130 (<reflink idref="bib10" id="ref1">10</reflink>), P = 0.200) but not for Rater Two (K‐S = 0.300 (<reflink idref="bib10" id="ref2">10</reflink>), P = 0.011). Therefore a conservative approach was deemed appropriate, and Spearman's correlation calculated for total scores between Rater One and Rater Two. At Stage 2, the assumptions of normality were met for both raters [K‐S = 0.214 (<reflink idref="bib7" id="ref3">7</reflink>), P = 0.200], therefore, a Pearson's correlation was calculated.</p> <p>A Wilcoxon Signed Rank Test revealed no statistically significant differences between the two raters' BSP‐QEII total scores at Stage 1, z = −1.268, P > 0.05, Spearman's r = 0.585, and a paired t‐test revealed no significant differences between the raters' BSP‐QEII total scores at Stage 2 [t = 0.70 (<reflink idref="bib6" id="ref4">6</reflink>), P = 0.518, Pearson's r = 0.843]. The percentage agreement, correlations and kappa for each of the BSP‐QEII items are shown in Tables 2 and 3.</p> <p>2 Inter‐rater agreement for Stage 1 between Raters One and Two for ( n  = 10 plans assessed)</p> <p> <ephtml> <table><tr><th>BSP‐QEII item</th><th>Agreement (%)</th><th>r*</th><th>P</th><th>Kappa†</th></tr><tr><td>A: Behaviour/s</td><td>70</td><td>0.22</td><td>0.545</td><td>0.21</td></tr><tr><td>B: Predictors of the Behaviour/s</td><td>90</td><td>0.67</td><td>0.035</td><td>0.62</td></tr><tr><td>C: Influencing factors</td><td>70</td><td>0.90</td><td>0.000</td><td>0.43</td></tr><tr><td>D: Environmental changes</td><td>80</td><td>0.55</td><td>0.098</td><td>0.47</td></tr><tr><td>E: Function/s of the behaviour/s</td><td>60</td><td>0.67</td><td>0.033</td><td>0.39</td></tr><tr><td>F: Replacement behaviour/s</td><td>70</td><td>0.52</td><td>0.077</td><td>0.43</td></tr><tr><td>G: Teaching strategies</td><td>60</td><td>0.34</td><td>0.532</td><td>0.30</td></tr><tr><td>H: Reinforcement</td><td>70</td><td>0.73</td><td>0.007</td><td>0.35</td></tr><tr><td>I: Reactive strategies</td><td>40</td><td>0.20</td><td>0.332</td><td>0.13</td></tr><tr><td>J: Goals and objectives</td><td>90</td><td>0.50</td><td>0.098</td><td>0.43</td></tr><tr><td>K: Team coordination</td><td>70</td><td>−0.17</td><td>0.645</td><td>−0.15</td></tr><tr><td>L: Communication</td><td>100</td><td>1.00</td><td>0.000</td><td>1.00</td></tr><tr><td>Mean</td><td>72.5</td><td>0.53</td><td /><td>0.41</td></tr></table> </ephtml> </p> <ulist> <item>4 * Spearman's correlations: 0.1–0.3 = weak, 0.4–0.6 = moderate, 0.7–0.9 = strong, 1 = perfect.</item> <item>5 Kappa: 0–0.2 = slight, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, 0.81–1 = almost perfect.</item> <item>6 BSP‐QEII, Behaviour Support Plan Quality Evaluation Guide II.</item> <item>3 Inter‐rater agreement for induction Stage 2 between Raters One and Two (for n  = 7 plans assessed)</item> </ulist> <p> <ephtml> <table><tr><th>BSP‐QEII item</th><th>Agreement (%)</th><th>r*</th><th>P</th><th>Kappa†</th></tr><tr><td>A: Behaviour/s</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>B: Predictors of the Behaviour/s</td><td>66.67</td><td>0.25</td><td>0.663</td><td>0.25</td></tr><tr><td>C: Influencing factors</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>D: Environmental changes</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>E: Function/s of the behaviour/s</td><td>50</td><td>0.45</td><td>0.373</td><td>0.22</td></tr><tr><td>F: Replacement behaviour/s</td><td>83.33</td><td>0.71</td><td>0.116</td><td>0.67</td></tr><tr><td>G: Teaching strategies</td><td>66.67</td><td>0.70</td><td>0.051</td><td>0.40</td></tr><tr><td>H: Reinforcement</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>I: Reactive strategies</td><td>50</td><td>0.26</td><td>0.537</td><td>0.25</td></tr><tr><td>J: Goals and objectives</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>K: Team coordination</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>L: Communication</td><td>100</td><td>1.00</td><td>0.000</td><td>1</td></tr><tr><td>Mean</td><td>83.33</td><td>0.78</td><td /><td>0.73</td></tr></table> </ephtml> </p> <ulist> <item>7 Pearson's correlations: 0.1–0.3 = weak, 0.4–0.6 = moderate, 0.7–0.9 = strong, 1 = perfect.</item> <item>8 Kappa: 0–0.2 = slight, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, 0.81–1 = almost perfect.</item> <item>9 BSP‐QEII, Behaviour Support Plan Quality Evaluation Guide II.</item> </ulist> <p>By surveying the above tables, it can be seen that the mean kappa score increased from a moderate agreement at Stage 1 to having substantial agreement at Stage 2. The most notable increases were for items A, C, D, H, J and K which reached 100% agreement. The results suggest evidence of marked improvement in assessing the quality of plans as the raters moved through each stage of the induction. Items B, E and I have slight agreement.</p> <hd id="AN0088263558-8">The time series study of behaviour support plan quality</hd> <p>The mean ratings for each of the BSP‐QEII items at Time One (2006/2007) and Time Two (2009/2010) are given in Fig. 1. The median BSP‐QEII total score on BSPs submitted to the OSP increased from Time One = 5/24 to Time Two = 11/24. Calculated total scores at Time One and Time Two violated the assumption of normality, therefore a Wilcoxon Signed Rank Test was used to assess for significant differences. A Wilcoxon Signed Rank Test revealed a statistically significant improvement in plan quality, z = −7.522, P = 0.00, with evidence of a relatively large effect size (r = 0.57).</p> <p>During the time series analyses IRR was checked on four occasions during the audit, so as to keep a check on ‘drift’. There were no statistically significant differences between the raters at these four time points, with inter‐rater agreement maintained at greater than 80%.</p> <hd id="AN0088263558-9">Preliminary investigation of organisational factors affecting behaviour support plan quality ...</hd> <p>Preliminary analyses revealed that the assumptions of normality had not been met for the distribution of organisational factors. Skewness and kurtosis were evident in combination with significant Kolmogrov–Smirnov statistics. Therefore a conservative approach was adopted in analysing these data. Where two conditions were assessed the Mann–Whitney U‐test was used, and where three or more conditions were assessed the Kruskal–Wallis Test was the statistical measure of choice. Results of organisational factors potentially affecting the quality of plans are presented in Table 4.</p> <p>4 Summary of descriptive statistics for organisational factors at Time Two</p> <p> <ephtml> <table><tr><th /><th>Mean</th><th>SD</th><th>Min</th><th>Max</th><th>Test result</th></tr><tr><td>Service sector</td><td /><td /><td /><td /><td /></tr><tr><td> Government</td><td>10.98</td><td>3.119</td><td>5</td><td>18</td><td>Z = −0.073, P > 0.05</td></tr><tr><td> Community sector</td><td>10.87</td><td>3.244</td><td>2</td><td>22</td><td /></tr><tr><td>Service type</td><td /><td /><td /><td /><td /></tr><tr><td> SSA</td><td>11.31</td><td>3.264</td><td>6</td><td>22</td><td>χ<sup>2</sup> = 10.230 (3), P = 0.017</td></tr><tr><td> Day support</td><td>9.80</td><td>3.293</td><td>2</td><td>16</td><td /></tr><tr><td> Respite</td><td>9.17</td><td>2.037</td><td>5</td><td>11</td><td /></tr><tr><td> CC</td><td>13.00</td><td>1.155</td><td>12</td><td>14</td><td /></tr><tr><td>Region</td><td /><td /><td /><td /><td /></tr><tr><td> Metro 1</td><td>11.50</td><td>3.695</td><td>6</td><td>22</td><td>χ<sup>2</sup> = 4.669 (7), P = 0.700</td></tr><tr><td> Metro 2</td><td>10.93</td><td>2.674</td><td>7</td><td>16</td><td /></tr><tr><td> Metro 3</td><td>10.67</td><td>3.366</td><td>2</td><td>16</td><td /></tr><tr><td> Rural 1</td><td>11.27</td><td>2.649</td><td>8</td><td>16</td><td /></tr><tr><td> Rural 2</td><td>9.50</td><td>2.510</td><td>7</td><td>14</td><td /></tr><tr><td> Rural 3</td><td>11.67</td><td>4.041</td><td>8</td><td>16</td><td /></tr><tr><td> Rural 4</td><td>11.75</td><td>0.957</td><td>11</td><td>13</td><td /></tr><tr><td> Rural 5</td><td>8.50</td><td>3.697</td><td>5</td><td>13</td><td /></tr><tr><td>BIST*</td><td /><td /><td /><td /><td /></tr><tr><td> None</td><td>9.97</td><td>2.662</td><td>2</td><td>16</td><td>χ<sup>2</sup> = 16.047 (2), P = 0.000</td></tr><tr><td> Consulted</td><td>13.33</td><td>3.440</td><td>8</td><td>22</td><td /></tr><tr><td> Author</td><td>11.50</td><td>2.507</td><td>8</td><td>14</td><td /></tr><tr><td>Format</td><td /><td /><td /><td /><td /></tr><tr><td> OSP</td><td>10.69</td><td>3.048</td><td>2</td><td>16</td><td>χ<sup>2</sup> = 2.023 (2), P > 0.05</td></tr><tr><td> OSP‐modified</td><td>12.12</td><td>3.551</td><td>8</td><td>22</td><td /></tr><tr><td> Agency's own</td><td>9.50</td><td>2.646</td><td>6</td><td>12</td><td /></tr></table> </ephtml> </p> <ulist> <item>10 Behaviour support team involvement.</item> <item>11 BIST, Behaviour Intervention Support Teams; CC, Congregate Care (older institutional facilities); SSA, Shared Supported Accommodation (shared houses in the community).</item> </ulist> <p>At Time Two, the Wilcoxon Signed Rank Test revealed no significant difference in plans by service sector, with both government and community sector organisations having a mean score of approximately 10. A Kruskal–Wallis Test revealed no significant differences in plans across regions or across format type.</p> <p>However, additional analysis conducted on the Time One data indicated the ‘Office of the Senior Practitioner (OSP) recommended guide’ as having a significant effect on the quality of plans, χ<sups>2</sups> = 8.260, P = 0.016. Post hoc tests reveal that plans developed using the OSP recommended guide format (Md = 8, n = 8) scored higher than when service providers used their own format (Md = 5, n = 78), z = −2.375, P = 0.018. These same differences were not evident at Time Two. Possible confounding effects will be discussed later.</p> <p>Differences in scores were evident between service types at Time Two. The Mann–Whitney U, post hoc test revealed significant differences in the overall BSP‐QEII score with shared supported accommodation (i.e. shared houses in the community) (Md = 11, n = 61) scoring higher than respite (Md = 9.5, n = 12), U = 207.50, z = −2.375, P = 0.02, r = 0.28. Furthermore, post hoc tests indicated that plans developed within congregate care settings (i.e. older institutional facilities) (Md = 13, n = 4) scored significantly higher than those developed by respite services (Md = 9.5, n = 12), U = 1.000, z = −2.818, P = 0.002, r = −0.7.</p> <p>At Time Two, a Kruskal–Wallis Test revealed significant differences in level of involvement by professional staff from the regional Behaviour Intervention Support Teams (BIST) across scores of the BSP's (no involvement, n = 58: consolation only, n = 21: authored plan, n = 8). Post hoc tests reveal plans which involved BIST consultation (Md = 13) scored significantly higher than plans which had no BIST involvement (Md = 10), z = −3.869, P = 0.000, r = 0.43.</p> <hd id="AN0088263558-10">Discussion</hd> <p>This study was designed to investigate the IRR of the BSP‐QEII, for use as an audit tool to assess the quality of BSPs for adults with ID in supported accommodation and other community‐based services. The study also investigated the utility of the BSP‐QEII for auditing the quality of plans submitted as a statutory requirement, and for monitoring changes in plan quality over time. The extent to which the BSP‐QEII could also identify variations in quality across different types of service and forms of service provision were also investigated.</p> <hd id="AN0088263558-11">Inter‐rater reliability</hd> <p>During the induction phase, raters attained a moderately strong level of agreement (r = 0.51, kappa = 0.43). Notably, during phase two of their induction process, the raters attained 83% agreement (r = 0.78, kappa = 0.73). Subsequently, IRR was maintained above 80% for the second part of the study, which involved rating a total of 174 BSPs (i.e. 87 pairs of plans, matched at two time periods). Caution is warranted when interpreting some of these findings because though many of the sub‐scale correlations were of a moderate to substantial strength some failed to reach statistical significance (α = 0.05). These issues could be associated with the relatively small sample size (i.e. the small number of plans assessed). The strength of the correlations overall, however, indicate desirable trends in the data with respect to overall agreement between raters.</p> <p>These findings compare favourably to those of an earlier pilot study in which raters with experience in reading, interpreting and applying BSPs to the direct support of adults with disability attained approximately 75% agreement, following review of 10 BSPs, and went on to attain approximately 90% agreement having reviewed some 30 BSPs (Webber et al. 2011b). Similarly, results from the present investigation replicate strong correlations reported for IRR found when the BSP‐QEII has been applied to the evaluation of BSPs in school settings (Kraemer et al. 2008; Cook et al. 2012). These results are promising as they indicate the BSP‐QEII scoring guide can be reliably applied to the evaluation of BSPs prepared for the support of adults in community‐based accommodation and other adult support settings, with some minimal staff training and mentorship.</p> <p>However, it should be noted that reliability varied across items. Generally, there was a high level of agreement about the presence or absence of information in BSPs relating to communication (i.e. about what information needed to be recorded, and how that information would be circulated among team members). However, raters appeared to have some difficulty agreeing the presence or absence of information relating to reactive strategies, information documenting the function of the behaviour, and what constituted teaching strategies. It may have been that BSP‐QEII items lacked sufficient detail to allow for objective scoring of plans prepared for use in adult services. However, in regards plan quality, of greater concern for practice is if two trained raters could not agree on these components being present it is highly probable that direct support staff required to use these BSPs in their daily work might experience similar difficulties identifying such components necessary to give direction to their work. Also of potential concern is that those responsible for formulating the plans were not able to clearly articulate these elements of good practice when preparing the BSPs.</p> <hd id="AN0088263558-12">The time series study of behaviour support plan quality</hd> <p>The BSP‐QEII was able to clearly distinguish between plans developed at the commencement of the new legislative requirements and associated sector‐wide educational strategies, and those developed 2 years later. Overall, plans developed during 2009/2010 scored significantly higher (46% scale maximum) than earlier plans developed in 2007–2008 (20% scale maximum). However, based on the BSP‐QEII mean scores, and according to the cut‐off scores specified by Browning‐Wright et al. (2003), all plans would be categorised as ‘weak’ in quality for both points in time. However, it needs to be acknowledged the cut‐off points for the quality categories, while consistent with those recommended by the authors of the BSP‐QEII, are relatively arbitrary, and further research is needed to clearly establish criteria to distinguish between plans of differing quality.</p> <p>Those areas of BSP quality which demonstrated the greatest improvements from time one to time two were with respect to documenting: (<reflink idref="bib1" id="ref5">1</reflink>) the factors influencing behaviour; (<reflink idref="bib2" id="ref6">2</reflink>) the predictors of the behaviour; and (<reflink idref="bib3" id="ref7">3</reflink>) the functions of the behaviour. Each of these areas shares in common the ability of staff to observe and document behaviour, and to formulate hypotheses as to why the behaviours might be occurring. These improvements appear consistent with the emphasis of educational strategies conducted by the OSP during 2008 and 2009. Those areas of BSP quality which showed the least improvement were: (<reflink idref="bib1" id="ref8">1</reflink>) specifying goals and objectives by which to measure the successful implementation of the plan (which in fact showed a small deterioration in quality between Time One and Time Two); (<reflink idref="bib2" id="ref9">2</reflink>) what needs to be recorded and how this information will be communicated among staff; (<reflink idref="bib3" id="ref10">3</reflink>) reactive strategies, together with how and when they are to be used; (<reflink idref="bib4" id="ref11">4</reflink>) the use of reinforcement strategies to reward alternative behaviours or the non‐occurrence of undesired behaviours; and (<reflink idref="bib5" id="ref12">5</reflink>) the teaching strategies that target the learning of new or alternative behaviours.</p> <p>Taken together, these results would strongly suggest ongoing education and training for staff is essential, if plans are to achieve the ultimate goal of improving service outcomes for the clients they are designed to support. Also, as noted above, further research is needed to clearly establish criteria to distinguish between plans of differing quality and, importantly, to establish links between these differing levels of quality in plans and the quality of client outcomes.</p> <hd id="AN0088263558-13">The investigation of organisational factors possibly influencing behaviour support plan ...</hd> <hd id="AN0088263558-14">Service sector and behaviour support plan format type</hd> <p>The quality of BSPs did not vary between government and community sector organisations, nor did they vary across administrative regions. These finding suggests that the disability service providers across sectors and regions have similar skills in both interpreting the legislative aspects and the planning aspects of behaviour support. This is consistent with the fact that the legislation was applicable to all disability service providers across all regions of the state. Furthermore, education programmes associated with the enactment of the new legislation had been delivered to both government and community agencies across all regions in 2007–2008.</p> <p>It had been suggested that use of standardised BSP formats, such as those developed by the OSP would serve to enhance the quality of BSP's. This assertion proved true for plans developed during Time One, at the enactment of the new legislation. However, the format guiding the development of BSPs failed to give rise to any significant difference in plan quality at Time Two. These results could indicate an overall improvement in knowledge and practice since the enactment of the new legislation and the associated educational programmes such that while in the past quality outcomes were dependent upon following an administrative format, educational initiatives had effectively improved the competency of service providers regardless of the BSP format they adopted. Such assertions would be consistent with those of Mansell et al. (2002) who report the intrinsic value of such educational programmes for increasing staff skills to address the complexities that occur when behavioural problems are present. However, it should be noted that the majority of BSPs at Time Two were either using the OSP planning guide or were substantially aligned with the OPS pro forma.</p> <hd id="AN0088263558-15">Type of support service</hd> <p>The quality of BSPs did vary across service type, with the highest quality plans evident for accommodation services, such as in congregate care settings and in share supported accommodation (i.e. group homes). These findings are consistent with the observation that staff in these settings typically have fewer clients on which they need to focus at any one time, and that they have the opportunity to develop more in‐depth knowledge of their clients over time, than might staff in day‐support or respite services. Furthermore, as accommodation services are typically considered the primary service provider, they are usually the service which takes primary responsibility for planning and coordinating behaviour support strategies. The current findings do, however, highlight the possible need to develop more educational and other (clinical) supports for staff in day‐support and respite services. The need to develop strategies to support quality behaviour support planning in respite services could be particularly important, given the added complexity that comes with the episodic provision of support, and staff having only limited opportunity to develop their understanding of the needs, preferences and priorities of the people they support. Furthermore, the development of strategies to facilitate the sharing of information and BSPs across different services than an individual accesses appears vital.</p> <p>When interpreting these findings it should however, be noted that the sample size for plans drawn from congregate care settings was relatively small compare to the other service types. Furthermore, those persons who were the subject of these plans were likely to have also been subject to some form of legal order (e.g. guardianship), and consequently were in receipt of a higher level of specialist service provision, and their BSPs were likely subject to more stringent reviews that would otherwise be typical for plans developed in other settings.</p> <hd id="AN0088263558-16">Behaviour Intervention Support Teams involvement</hd> <p>Higher quality plans were evident where specialist service providers (i.e. BIST) had been involved. These results are consistent with the assertion above concerning why plans formulated for clients in Congregate Care settings were of a higher quality than other plans. Furthermore, these results are consistent with the findings of Cook et al. (2007) which reports that plans developed by professionals trained in BSP development were more likely to produce adequate plans, as measured by the BSP‐QEII, when compared to those developed by others. Here it is noteworthy that there was a trend for plans directly authored by BIST practitioners to be of a lesser quality than those where BIST practitioners had only provided consultation. However, there were no statistically significant differences evident in the data. The current results could have been attributed to the relatively small number of such plans in the current analysis. Further investigation with a larger sample could yield a more robust analysis.</p> <hd id="AN0088263558-17">Limitations of the current study, and future directions for research</hd> <p>The IRR reported in the first part of this study only involved two raters, as only two raters were available at the time. A future study might consider comparisons involving additional raters, for which purposes Kendall's W could be substituted for Kappa. Furthermore, only 17 assessments were considered in the first part of the study. However, this number was sufficient to establish the level of experience necessary for those using the BSP‐QEII for audit purposes. Also, IRR was checked on a further four occasions during the second part of the study, involving a further 174 plans, the results of which indicated IRR to remain above 80% agreement.</p> <p>Each rater alternated rating of Time One and Time Two for each pair of BSPs they evaluated. Raters were aware of the dates of the BSPs and it is possible that this information influenced their ratings. Future studies should consider providing raters with plans without dates to lessen the likelihood of any bias occurring. It should be noted that conclusions made in regards to quality categories of the plans were based on the criteria recommended by the authors of the BSP‐QEII. However, given there is limited evidence to support the current criteria for these categories, these categories are fairly arbitrary and further research is required. Such research could potentially link BSP quality ratings to client outcomes, and in so doing would provide a more robust system of quality categorisation for BSPs. Finally, it should be noted that the category describing BIST involvement by way of ‘consultation’ included some situations in which plans had been developed with reference to clinical reports that were several years old. Future studies should account for different forms of specialist consultation.</p> <hd id="AN0088263558-18">Conclusions</hd> <p>The results of the current study support the reliability of the BSP‐QEII for the evaluation of BSP's developed in government and community‐based support services for adults with ID. There is some evidence that the BSP‐QEII can discriminate between plans developed over time, and under different circumstances. However, it must be acknowledged that the various changes over time to policy, together with administrative and clinical strategies, explored in relation to variations in plan quality, appeared only to have a modest impact on BSP‐QEII ratings. The current data do not allow for speculation as to why this might be so, but further research is clearly warranted. Furthermore, the current results suggest that, according to the BSP‐QEII criteria, BSPs in the jurisdiction in which this study was conducted remain of a relatively low quality, despite legislative, policy and educational endeavours, and that further efforts are required to achieve those standards prescribed in the BSP‐QEII.</p> <ref id="AN0088263558-19"> <title>References</title> <blist> <bibl id="bib1" idref="ref5" type="bt">1</bibl> <bibtext>Allen D. & Hill‐Tout J. ( 1998 ) A day in the life: day activities for people with intellectual disability and challenging behaviour in two English counties. Journal of Applied Research in Intellectual Disabilities 12, 30 – 45. </bibtext> </blist> <blist> <bibl id="bib2" idref="ref6" type="bt">2</bibl> <bibtext>Anastasi A. & Urbina S. P. ( 1997 ) Psycholgical Testing, Seventh International Edition. Prentics Hall, Upper Saddle River, NJ. </bibtext> </blist> <blist> <bibl id="bib3" idref="ref7" type="bt">3</bibl> <bibtext>Australian Psychological Society [APS] ( 2011 ) Evidence‐Based Guidelines to Reduce the Need for Restrictive Practices in the Disability Sector. APS, Melbourne. Available at: <ulink href="http://www.psychology.org.au/Assets/Files/Restrictive-Practices-Guidelines-for-Psychologists.pdf">http://www.psychology.org.au/Assets/Files/Restrictive-Practices-Guidelines-for-Psychologists.pdf</ulink> (retrieved 1 December 2011). </bibtext> </blist> <blist> <bibl id="bib4" idref="ref11" type="bt">4</bibl> <bibtext>Ball T., Bush A. & Emerson E. ( 2004 ) Psychological Interventions for Severely Challenging Behaviors Shown by People with Learning Disabilities. The British Psychological Association, Leicester. </bibtext> </blist> <blist> <bibl id="bib5" idref="ref12" type="bt">5</bibl> <bibtext>Bird F. L. & Luiselli J. K. ( 2000 ) Positive behavioural support of adults with developmental disabilities: assessment of long term adjustment and habilitation following restrictive treatment histories. Journal of Behavior Therapy and Experimental Psychiatry 31, 5 – 19. </bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext>Blood E. & Neel R. S. ( 2007 ) From FBA to implementation: a look at what is actually being delivered. Education and Treatment of Children 30, 67 – 80. </bibtext> </blist> <blist> <bibl id="bib7" idref="ref3" type="bt">7</bibl> <bibtext>Browning‐Wright D., Saren D. & Mayer G. R. ( 2003 ) The behaviour support plan‐quality evaluation guide. Available at: <ulink href="http://www.pent.ca.gov">http://www.pent.ca.gov</ulink> (retrieved 1 December 2011). </bibtext> </blist> <blist> <bibl id="bib8" type="bt">8</bibl> <bibtext>Browning‐Wright D., Mayer G. R., Cook C. R., Crews S. D., Kraemer B. R. & Gale B. ( 2007 ) A preliminary study on the effects of training using Behaviour Support Plan Quality Evaluation Guide (BSP‐QE) to improve positive behavioural support plans. Education and Treatment of Children 30, 89 – 106. </bibtext> </blist> <blist> <bibl id="bib9" type="bt">9</bibl> <bibtext>Carr E. G. & Horner R. H. ( 2007 ) The expanding vision of positive behavior support. Journal of Positive Behavior Interventions 9, 3 – 14. </bibtext> </blist> <blist> <bibl id="bib10" idref="ref1" type="bt">10</bibl> <bibtext>Carr E. G., Horner R. H., Turnball A. P., McLaughlin D. M., McAtee M. L., Smith C. E. et al. ( 1999 ) Positive behavior support for people with developmental disabilities: a research synthesis. Washington, DC: American Association on Mental Retardation. A report requested from the United States Department of Education, Office of Special Education Programs. </bibtext> </blist> <blist> <bibl id="bib11" type="bt">11</bibl> <bibtext>Cook C. R., Crews S. D., Browning‐Wright D. B., Mayer G. R., Gale B., Kraemer B. et al. ( 2006 ) Establishing and evaluating the substantive adequacy of positive behavioral support plans. Journal of Behavioral Education 16, 191 – 206. </bibtext> </blist> <blist> <bibl id="bib12" type="bt">12</bibl> <bibtext>Cook C. R., Crews D., Browning‐Wright D., Mayer R., Gale B., Kraemer B. et al. ( 2007 ) Establishing and evaluating the substantive adequacy of positive behavior support plans. Journal of Behavioral Education 16, 191 – 206. </bibtext> </blist> <blist> <bibl id="bib13" type="bt">13</bibl> <bibtext>Cook C. R., Mayer G. R., Browning‐Wright D., Kraemer B., Wallace M. D., Dart E. et al. ( 2012 ) Exploring the link among behaviour intervention plans, treatment integrity and student outcomes under natural conditions. The Journal of Special Education 46, 3 – 16. </bibtext> </blist> <blist> <bibl id="bib14" type="bt">14</bibl> <bibtext>Cooper S. A., Smiley E., Allan L. M., Jackson A., Finlayson J., Mantry D. et al. ( 2009 ) Adults with intellectual disabilities: prevalence, incidence and remission of self‐injurious behaviour, and related factors. Journal of Intellectual Disability Research 53, 200 – 16. </bibtext> </blist> <blist> <bibl id="bib15" type="bt">15</bibl> <bibtext>Crocker A. G., Mercier C., Lachapelle Y., Brunet A., Morin D. & Roy M. E. ( 2006 ) Prevalence and types of aggressive behaviour among adults with ID. Journal of Intellectual Disability Research 50, 652 – 61. </bibtext> </blist> <blist> <bibl id="bib16" type="bt">16</bibl> <bibtext>Emerson E. & Einfeld S. ( 2011 ) Challenging Behaviour, 3rd edn. Cambridge University Press, Cambridge. </bibtext> </blist> <blist> <bibl id="bib17" type="bt">17</bibl> <bibtext>Emerson E., Kiernan C., Alborz A., Reeves D., Mason H., Swarbrick R. et al. ( 2001 ) The prevalence of challenging behaviours: a total population study. Research in Developmental Disabilities 22, 77 – 93. </bibtext> </blist> <blist> <bibl id="bib18" type="bt">18</bibl> <bibtext>Feldman M. A., Atkinson L., Foti‐Gervais L. & Condillac R. ( 2004 ) Formal versus informal interventions for challenging behaviours in persons with intellectual disabilities. Journal of Intellectual Disability Research 48, 60 – 8. </bibtext> </blist> <blist> <bibl id="bib19" type="bt">19</bibl> <bibtext>Gwet K. ( 2010 ) Handbook of Inter‐Rater Reliability, 2nd edn. Advanced Analytics, LLC, Gaithersburg, MD. </bibtext> </blist> <blist> <bibl id="bib20" type="bt">20</bibl> <bibtext>Horner R. H., Sugai G., Todd A. W. & Lewis‐Palmer T. ( 2000 ) Elements of behavior support plans: a technical brief. Exceptionality: A Special Education Journal 8, 205 – 15. </bibtext> </blist> <blist> <bibl id="bib21" type="bt">21</bibl> <bibtext>Kraemer B. R., Cook C. R., Browning‐Wright D., Mayer G. R. & Wallace M. D. ( 2008 ) Effects of training on the use of the Behavior support plan quality evaluation guide with autism educators. Journal of Positive Behavior Interventions 10, 179 – 89. </bibtext> </blist> <blist> <bibl id="bib22" type="bt">22</bibl> <bibtext>Landis J. R. & Koch G. G. ( 1977 ) The measurement of observer agreement for categorical data. Biometrics 33, 159 – 74. </bibtext> </blist> <blist> <bibl id="bib23" type="bt">23</bibl> <bibtext>Mansell J., Ashman B., MacDonald S. & Beadle‐Brown J. ( 2002 ) Residential care in the community for adults with intellectual disability: needs, characteristics and services. Journal of Intellectual Disability Research 46, 625 – 33. </bibtext> </blist> <blist> <bibl id="bib24" type="bt">24</bibl> <bibtext>McClean B., Grey I. M. & McCracken M. ( 2007 ) An evaluation of positive behavioural support for people with very severe challenging behaviours in community‐based settings. Journal of Intellectual Disabilities 11, 281 – 301. </bibtext> </blist> <blist> <bibl id="bib25" type="bt">25</bibl> <bibtext>McVilly K., Webber L., Sharp G. & Paris M. ( 2012 ) The content validity of the Behaviour Support Plan Quality Evaluation tool (BSP‐QEII) and its potential application in accommodation and day‐support services for adults with intellectual disability. Journal of Intellectual Disability Research (in press). </bibtext> </blist> <blist> <bibl id="bib26" type="bt">26</bibl> <bibtext>Phillips L., Wilson L. & Wilson E. ( 2010 ) Assessing behaviour support plans for people with intellectual disability before and after the Victorian Disability Act, 2006. Journal of Intellectual & Developmental Disability 35, 9 – 13. </bibtext> </blist> <blist> <bibl id="bib27" type="bt">27</bibl> <bibtext>Webber L., McVilly K., Stevenson E. & Chan J. ( 2010 ) Restrictive interventions for people with intellectual disability exhibiting challenging behaviours of concern: analyses of a population database. Journal of Applied Research in Intellectual Disabilities 35, 199 – 206. </bibtext> </blist> <blist> <bibl id="bib28" type="bt">28</bibl> <bibtext>Webber L., McVilly K., Fester T. & Chan J. ( 2011a ) Factors influencing quality of behaviour support plans and the impact of plan quality of on restrictive intervention use. International Journal of Positive Behavioural Support 1, 24 – 31. </bibtext> </blist> <blist> <bibl id="bib29" type="bt">29</bibl> <bibtext>Webber L., McVilly K., Fester T. & Zazelis T. ( 2011b ) Assessing behaviour support plans for Australian adults with intellectual disability using the ‘Behavior Support Plan Quality Evaluation II’ (BSP‐QE II). Journal of Intellectual and Developmental Disability 36, 1 – 5. </bibtext> </blist> </ref> <p>Graph: 1 Behaviour Support Plan Quality Evaluation Guide II (BSP‐QEII) mean ratings for each of the items at Time One and Time Two.</p> <p>Graph: image%5fn/jir1603%5ff1.gif</p> <aug> <p>By K. McVilly; L. Webber; M. Paris and G. Sharp</p> </aug>
Header DbId: eric
DbLabel: ERIC
An: EJ1014798
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Reliability and Utility of the Behaviour Support Plan Quality Evaluation Tool (BSP-QEII) for Auditing and Quality Development in Services for Adults with Intellectual Disability and Challenging Behaviour
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22McVilly%2C+K%2E%22">McVilly, K.</searchLink><br /><searchLink fieldCode="AR" term="%22Webber%2C+L%2E%22">Webber, L.</searchLink><br /><searchLink fieldCode="AR" term="%22Paris%2C+M%2E%22">Paris, M.</searchLink><br /><searchLink fieldCode="AR" term="%22Sharp%2C+G%2E%22">Sharp, G.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Intellectual+Disability+Research%22"><i>Journal of Intellectual Disability Research</i></searchLink>. Aug 2013 57(8):716-727.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2013
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Behavior+Problems%22">Behavior Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Behavior+Modification%22">Behavior Modification</searchLink><br /><searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br /><searchLink fieldCode="DE" term="%22Mental+Retardation%22">Mental Retardation</searchLink><br /><searchLink fieldCode="DE" term="%22Day+Programs%22">Day Programs</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Validity%22">Test Validity</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Planning%22">Planning</searchLink><br /><searchLink fieldCode="DE" term="%22Quality+Control%22">Quality Control</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Interrater+Reliability%22">Interrater Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Teamwork%22">Teamwork</searchLink><br /><searchLink fieldCode="DE" term="%22Predictor+Variables%22">Predictor Variables</searchLink><br /><searchLink fieldCode="DE" term="%22Environmental+Influences%22">Environmental Influences</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement%22">Reinforcement</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Responses%22">Responses</searchLink><br /><searchLink fieldCode="DE" term="%22Goal+Orientation%22">Goal Orientation</searchLink><br /><searchLink fieldCode="DE" term="%22Interpersonal+Communication%22">Interpersonal Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Audits+%28Verification%29%22">Audits (Verification)</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Australia%22">Australia</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/j.1365-2788.2012.01603.x
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0964-2633
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Having an objective means of evaluating the quality of behaviour support plans (BSPs) could assist service providers and statutory authorities to monitor and improve the quality of support provided to people with intellectual disability (ID) who exhibit challenging behaviour. The Behaviour Support Plan Quality Evaluation Guide II (BSP-QEII) was developed to monitor and assess BSPs prepared by teachers to support children with disability in the school system. This study investigated the application of the BSP-QEII to the assessment of BSPs for adults with ID in community support services. Method: The inter-rater reliability of the BSP-QEII was assessed. The utility of the BPS-QEII was then investigated with reference to a time series study of matched pairs of BSPs, developed for the same clients over a period of approximately 3 years. Differences in plan quality measured across a number of service and systemic variables were also investigated. Results: The BSP-QEII was found to have good inter-rater reliability and good utility for audit purposes. It was able to discriminate changes in plan quality over time. Differences in plan quality were also evident across different service types, where specialist staff had or had not been involved, and in some instances where a statutory format for the plan had or had not been used. There were no differences between plans developed by government and community sector agencies, nor were there any regional differences across the jurisdiction. Conclusions: The BSP-QEII could usefully be adopted as an audit tool for measuring the quality of BSPs for adults with ID. In addition to being used for research and administrative auditing, the principles underpinning the BSP-QEII could also be useful to guide policy and educational activities for staff in community based services for adults with ID. (Contains 4 tables and 1 figure.)
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Ref
  Label: Number of References
  Group: RefInfo
  Data: 29
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2014
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1014798
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1014798
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/j.1365-2788.2012.01603.x
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 716
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Behavior Problems
        Type: general
      – SubjectFull: Behavior Modification
        Type: general
      – SubjectFull: Adults
        Type: general
      – SubjectFull: Mental Retardation
        Type: general
      – SubjectFull: Day Programs
        Type: general
      – SubjectFull: Test Validity
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Planning
        Type: general
      – SubjectFull: Quality Control
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Interrater Reliability
        Type: general
      – SubjectFull: Teamwork
        Type: general
      – SubjectFull: Predictor Variables
        Type: general
      – SubjectFull: Environmental Influences
        Type: general
      – SubjectFull: Reinforcement
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Responses
        Type: general
      – SubjectFull: Goal Orientation
        Type: general
      – SubjectFull: Interpersonal Communication
        Type: general
      – SubjectFull: Audits (Verification)
        Type: general
      – SubjectFull: Australia
        Type: general
    Titles:
      – TitleFull: Reliability and Utility of the Behaviour Support Plan Quality Evaluation Tool (BSP-QEII) for Auditing and Quality Development in Services for Adults with Intellectual Disability and Challenging Behaviour
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: McVilly, K.
      – PersonEntity:
          Name:
            NameFull: Webber, L.
      – PersonEntity:
          Name:
            NameFull: Paris, M.
      – PersonEntity:
          Name:
            NameFull: Sharp, G.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Type: published
              Y: 2013
          Identifiers:
            – Type: issn-print
              Value: 0964-2633
          Numbering:
            – Type: volume
              Value: 57
            – Type: issue
              Value: 8
          Titles:
            – TitleFull: Journal of Intellectual Disability Research
              Type: main
ResultId 1