View in EDS HTML Full Text PDF Full Text

Evaluating the Relationship between Performance on a Multiple-Choice Examination and Common ABA-Based Procedures

Saved in:
Bibliographic Details
Title: Evaluating the Relationship between Performance on a Multiple-Choice Examination and Common ABA-Based Procedures
Language: English
Authors: Leaf, Justin B., Cihon, Joseph H., Ferguson, Julia L., Milne, Christine, Leaf, Ronald, McEachin, John
Source: Focus on Autism and Other Developmental Disabilities. Jun 2023 38(2):113-123.
Availability: SAGE Publications and Hammill Institute on Disabilities. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 11
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Descriptors: Applied Behavior Analysis, Multiple Choice Tests, Certification, Professional Personnel, Scores, Correlation, Autism Spectrum Disorders, Intervention, Children, Behavior Problems, Clinics, Reinforcement, Conditioning
DOI: 10.1177/10883576221110170
ISSN: 1088-3576
1538-4829
Abstract: To ensure the quality of training, skills developed and maintained, and expected standards for behavior analysts, several systems have been put into place across the years. Of these systems, perhaps the most widespread was the development of the Behavior Analyst Certification Board in 1998. Once all requirements are met, the final determinative step in obtaining certification is a multiple-choice examination. The use of a multiple-choice examination as the final determinative step in obtaining behavior analytic certification may create some concerns for behavior analysts. Identification of a positive correlation between the multiple-choice examination and performance may help ameliorate these concerns. The purpose of this study was to evaluate any potential correlations between scores on a popular commercially available prep examination and basic applied behavior analysis (ABA)--based autism intervention procedures with 25 professionals. Overall, the results indicated that multiple-choice examination scores were not significantly correlated with any assessments of performance.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1377943
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwGkoH9Mwv8ZaMjHnnjcn3UMAAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDa233wK2A3GxbYDxAIBEICBm20vPb4Qfqq3jrMgq2t3BV7Fh83lMJKwSrd4FmQubve2I8svOfTmY8XIoYbGLx7KbI9GyJLbqQum921qrYOFYxxfyYbRL2wRN1GC2zpNdGf7L76x4A_zH9LUuKq6LK7M9dEM-Yqi5CD6jySmVVHL9YDkP2yR80kwjTBPRz5ZvHvTjpdLYBICgKjxH0KUJYIcjLZ3kVTpI8XPvaK1
Text:
  Availability: 1
  Value: <anid>AN0163683943;fdd01jun.23;2023May16.03:15;v2.2.500</anid> <title id="AN0163683943-1">Evaluating the Relationship Between Performance on a Multiple-Choice Examination and Common ABA-Based Procedures </title> <p>To ensure the quality of training, skills developed and maintained, and expected standards for behavior analysts, several systems have been put into place across the years. Of these systems, perhaps the most widespread was the development of the Behavior Analyst Certification Board in 1998. Once all requirements are met, the final determinative step in obtaining certification is a multiple-choice examination. The use of a multiple-choice examination as the final determinative step in obtaining behavior analytic certification may create some concerns for behavior analysts. Identification of a positive correlation between the multiple-choice examination and performance may help ameliorate these concerns. The purpose of this study was to evaluate any potential correlations between scores on a popular commercially available prep examination and basic applied behavior analysis (ABA)–based autism intervention procedures with 25 professionals. Overall, the results indicated that multiple-choice examination scores were not significantly correlated with any assessments of performance.</p> <p>Keywords: certification; staff assessment; staff training; training</p> <p>To ensure behaviorally based interventions for autistics/individuals diagnosed with autism spectrum disorder (ASD) are of sufficient quality and comprehensiveness, they should be implemented with a high degree of quality ([<reflink idref="bib6" id="ref1">6</reflink>]) from well-trained professionals ([<reflink idref="bib1" id="ref2">1</reflink>]). As a result, there have been efforts to help consumers (e.g., autistics/individuals diagnosed with ASD, parents, and family members), third-party payers (e.g., insurance companies), educators, and society as a whole to identify professionals who have met a specific level of competency. More specifically, certifying bodies and behavior analytic certifications have been developed to help ensure quality control, develop standards of competency, and help protect consumers from individuals who are not qualified ([<reflink idref="bib19" id="ref3">19</reflink>]; [<reflink idref="bib21" id="ref4">21</reflink>]). Most notably, the creation of the Behavior Analyst Certification Board (BACB) was, in part, developed as a method to help foster quality and consistency across practitioners. The creation of the BACB has assisted consumers in identifying competent and qualified behavior analysts and has helped established a professional identity that permits third-part billing.</p> <p>The success of the BACB certification is evident by a continual increase in the number of certified practitioners (see [<reflink idref="bib5" id="ref5">5</reflink>]) across the last 30 years. In 1999, there were only 30 Board Certified Behavior Analysts (BCBAs) and Board Certified Assistant Behavior Analysts (BCaBAs) in the world. As of July 1, 2021, there are over 53,000 ([<reflink idref="bib4" id="ref6">4</reflink>]). The BACB has also added a paraprofessional-level certification for those who are responsible for direct implementation of behavior analytic services known as the Registered Behavior Technician (RBT). As of July 1, 2021, there are over 100,000 RBTs ([<reflink idref="bib4" id="ref7">4</reflink>]). Taken together, there are now over 156,000 practitioners credentialed by the BACB, the majority of whom work within the autism, or related, field ([<reflink idref="bib4" id="ref8">4</reflink>]).</p> <p>The increase in the number of certified behavior analysts is not surprising, given the contingencies operating in the field of ASD. For example, professionals are commonly required to hold a behavior analytic certification to be eligible for third party billing. Meaning that for autistics/individuals diagnosed with ASD to receive behavioral intervention, they must be implemented and supervised by those who are certified. Furthermore, it is not uncommon for BCBAs, BCaBAs, and RBTs to provide intervention and/or supervision for autistics/individuals diagnosed with ASD within school districts. Finally, certification from the BACB is also commonly included in job posting for various behavior analytic positions within the autism field. As a result, certification and practice within ABA-based autism intervention have become fundamentally linked. As such, issues around certification do not only affect the field of behavior analysis but all autistics/individuals diagnosed with ASD and those associated with ASD.</p> <p>Although, the BACB is not an autism certification, the majority of certified professionals are working in some capacity with autistics/individuals diagnosed with ASD ([<reflink idref="bib4" id="ref9">4</reflink>]). The BACB should be lauded for their efforts to define standards within behavior analysis, protect consumers, and help ensure quality intervention for autistics/individuals diagnosed with ASD. Nonetheless, there has been some professional discourse within the literature related to concerns with specific certification requirements outlined by the BACB (e.g., [<reflink idref="bib7" id="ref10">7</reflink>]; [<reflink idref="bib12" id="ref11">12</reflink>], [<reflink idref="bib11" id="ref12">11</reflink>]). These concerns have related to the scope of task lists, transparency with respect to the use of subject matter experts, and the number of required experience and supervision hours ([<reflink idref="bib12" id="ref13">12</reflink>], [<reflink idref="bib11" id="ref14">11</reflink>]; [<reflink idref="bib15" id="ref15">15</reflink>]; [<reflink idref="bib17" id="ref16">17</reflink>]).</p> <p>Another concern is related to the final step in obtaining certification. Currently, there are several components required to obtain behavior analytic certification, including obtaining a specific degree, completing a coursework sequence, and meeting certain training requirements, as well as passing competency assessments. While some components required to obtain behavior analytic certification may include direct application of ABA-based procedures (e.g., practicum, supervised field work), the final step in obtaining certification involves taking and passing an examination consisting of multiple-choice questions (MCQs). In other words, an individual may obtain the appropriate degree, meet supervision requirements, and pass a competency assessment but fail the examination, which would result in not attaining certification. In this light, the MCQ examination becomes the final, determinative step in obtaining certification. This is not surprising, given the use of MCQ examinations are (<reflink idref="bib1" id="ref17">1</reflink>) standard practice in the certification process, (<reflink idref="bib2" id="ref18">2</reflink>) "align with best practices in credentialing industry" ([<reflink idref="bib7" id="ref19">7</reflink>], p. 164), and (<reflink idref="bib3" id="ref20">3</reflink>) "are legally defensible" ([<reflink idref="bib7" id="ref21">7</reflink>], p. 164).</p> <p>The use of an MCQ examination as the final determinative step in obtaining behavior analytic certification may create some concerns for anyone associated with ASD. For instance, [<reflink idref="bib2" id="ref22">2</reflink>] highlighted that behavior analysts are more concerned about "what subjects can be brought to do rather than what they can be brought to say" (p. 93). MCQ examination appears, on the surface, to be an assessment of what potential certificants are able to say rather than what they can do. Therefore, the generality of MCQ examination appears to be limited to assessment of responding on the examination itself and is unlikely to be associated with performance in other contexts (e.g., performance of common ABA-based procedures with autistics/individuals diagnosed with ASD). In other words, passing an examination is not indicative if a BCBA can implement or supervise autistics/individuals diagnosed with ASD with any level of competency, rather it is indicative they can effectively answer questions about implementation and supervision of behavior analytic practices.</p> <p>Given the commonality of say–do correspondence failures ([<reflink idref="bib16" id="ref23">16</reflink>]), an evaluation of any correlation between what candidates can <emph>say</emph> (i.e., responses on the MCQ examination) and what they can <emph>do</emph> (i.e., implementation of ABA-based procedures) may prove fruitful. Identification of a positive correlation between the MCQ examination and performance of ABA-based procedures could be very useful to the field of ASD as it may demonstrate that current certification standards correspond with certificants' skillset with respect to the implementation of ABA-based procedures. Identification of a negative correlation between the MCQ examination and performance of ABA-based procedures would indicate that possessing a certification may not be an indicator if a professional can implement ABA-based procedures effectively. An evaluation of this sort may also be critical for agencies and school districts that hire behavior analysts. For instance, although competency in various ABA-based procedures is commonly required by agencies and school districts, it may be uncommon for employers to assess competency in behavior analytic procedures prior to or at the onset of employment. Rather, certification permits a level of confidence that professionals have met a standard set of minimum requirements to obtain and maintain the credential. A confidence that might not be warranted and which may result in professionals implementing procedures to an unacceptable quality.</p> <p>An evaluation of any possible correlations between the MCQ examination and performance of ABA-based procedures may also contribute to the literature on adult learning, assessment, and professional development. It is not uncommon for professional development to involve some form of didactic training followed by an assessment in the form of a quiz (e.g., MCQ examination). This is likely done in an effort to train as many people as possible, while having a standardized system of measurement. Identifying a positive correlation between an MCQ examination and performance could provide preliminary levels of validity to this method of training and testing. Relatedly, the results of such a study could have implications for the behavior analytic literature on adult learning. Specifically, identifying the conditions under which performance on an MCQ examination is correlated with actually performance could inform more effective and efficient training and assessment methods for adult learners.</p> <p>To date, there have not been any published, empirical studies evaluating the level of say–do correspondence between MCQ examination performance to obtain certification and actual performance on ABA-based procedures. Therefore, the purpose of this study was to investigate if there was a relationship between performance on an analogue of the BCBA examination and performance of common ABA-based procedures used with autistics/individuals diagnosed with ASD. Like many certifications, the test questions on the BCBA exam are proprietary; thus, one way to evaluate a correlation between this exam and performance of ABA-based procedures could be using scores on an exam that is highly correlated with scores on the BCBA exam. As such, we compared scores on a commercially available BCBA examination prep tool that has documented a high degree of predictiveness of passing the BCBA examination (i.e., Behavior Development Solutions [see [<reflink idref="bib6" id="ref24">6</reflink>]]; [<reflink idref="bib5" id="ref25">5</reflink>].) to scores on assessments of participant implementation of discrete trial teaching (DTT), conditioning reinforcers, and general implementation of behavior change procedures with autistics/individuals diagnosed with ASD.</p> <hd id="AN0163683943-2">Method</hd> <p></p> <hd id="AN0163683943-3">Participants</hd> <p>A convenience sample of participants was recruited from staff working at a private agency that provides behavioral intervention for children diagnosed with ASD. All participants were informed that participation in this study was voluntary, and none of their data would be shared with their employer, and no incentives were provided for participation. As such, participation in this study would not affect their employment status at the agency. There were four inclusion criteria for participants. First, participants had to have at least 1 year of experience with behavioral intervention as it applies to autistics/individuals diagnosed with ASD. Second, participants had to have at least a bachelor's degree. Third, participants could not be a certified behavior technician or behavior analyst. Fourth, participants had to consent to be part of the study. Twenty-five participants volunteered to participate in the study. Table 1 provides the participants' age, education level, and experience with ABA. The mean staff age was 33 years (range, 23–53 years) with the majority of staff having over 4 years of experience with ABA (i.e., 17 participants). Nineteen of the participants had a bachelor's degree and six had a master's degree. Of the 19 participants who had a bachelor's degree, 10 were currently pursuing a master's degree in behavior analysis.</p> <p>Graph</p> <p>Table 1. Participant Demographics for Study on Multiple Choice Examination Scores.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /><col align="char" char="." /><col align="char" char="." /></colgroup><thead><tr><th align="left">Participant</th><th align="center">Age</th><th align="center">Education level</th><th align="center">Experience</th></tr></thead><tbody><tr><td>A</td><td>27</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>B</td><td>34</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>C</td><td>33</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>D</td><td>23</td><td>Bachelor's degree</td><td><3 years</td></tr><tr><td>E</td><td>30</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>F</td><td>32</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>G</td><td>31</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>H</td><td>26</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>I</td><td>53</td><td>Master's degree</td><td>>3 years</td></tr><tr><td>J</td><td>36</td><td>Master's degree</td><td>>3 years</td></tr><tr><td>K</td><td>24</td><td>Bachelor's degree</td><td><3 years</td></tr><tr><td>L</td><td>28</td><td>Master's degree</td><td>>3 years</td></tr><tr><td>M</td><td>34</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>N</td><td>37</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>O</td><td>29</td><td>Master's degree</td><td>>3 years</td></tr><tr><td>P</td><td>29</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>Q</td><td>32</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>R</td><td>29</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td>>3 years</td></tr><tr><td>S</td><td>48</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>T</td><td>26</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>U</td><td>33</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>V</td><td>29</td><td>Bachelor's degree</td><td>>3 years</td></tr><tr><td>W</td><td>33</td><td>Bachelor's degree<xref ref-type="table-fn" rid="tfn1">a</xref></td><td><3 years</td></tr><tr><td>X</td><td>45</td><td>Master's degree</td><td>>3 years</td></tr><tr><td>Y</td><td>38</td><td>Master's degree</td><td>>3 years</td></tr></tbody></table> </ephtml> </p> <p>1 Participants were currently pursuing their master's degree in behavior analysis.</p> <hd id="AN0163683943-4">Setting</hd> <p>The study took place in a private, not-for-profit clinic that provides behavioral intervention for children diagnosed with ASD. The study took place in various rooms within the clinic. Each room had a child-sized table, child-sized chairs, instructional materials, reinforcing items/materials, and various office materials (e.g., paper, pencils, and pens).</p> <hd id="AN0163683943-5">General Procedure</hd> <p>Each participant was exposed to four assessments: (a) MCQ examination, (b) DTT, (c) conditioning reinforcers, and (d) general behavioral intervention. Each participant first completed the MCQ examination, followed by the DTT and conditioning reinforcer assessments, and concluded with the general behavioral intervention assessment. All assessments were videotaped for scoring afterwards. The scores on the MCQ examination were then compared to the other assessments in order to evaluate any possible relationships between the MCQ examination scores and the other implementation assessments.</p> <hd id="AN0163683943-6">Multiple choice questions examination</hd> <p>The MCQ examination was comprised of testing modules from a popular commercially available BCBA examination prep tool (i.e., Behavior Development Solutions; [<reflink idref="bib5" id="ref26">5</reflink>].). [<reflink idref="bib5" id="ref27">5</reflink>] was selected based on data available on correlations between the modules and pass rates on the BCBA examination. Surveys conducted by [<reflink idref="bib5" id="ref28">5</reflink>] "have yielded a pass rate between 91% and 97% (i.e., 96%, 97%, 91%, and 97% in 2013, 2015, 2017/2018, and 2019, respectively)" (para. 1). That is, completion of all the Behavior Development Solutions learning modules at 100% correct is highly correlated with successful first-time pass rates on the BCBA exam. As such, if participants score 100% on the MCQ examination used within this study, it is likely that they would also pass the BCBA exam. It should be noted, that while [<reflink idref="bib5" id="ref29">5</reflink>] is designed to teach, the educational part was excluded in this study and only the pre/post examination testing modules were used. As a result, the scores reported by participants on the MCQ examination are not reflective of the effectiveness of Behavior Development Solutions' training modules.</p> <p>The MCQ examination covered 12 different content areas: (a) measurement 1, (b) experimental design, (c) behavior change considerations, (d) fundamental elements of behavior change, (e) specific behavior change procedures, (f) behavior change systems, (g) identification of the problem, (h) measurement 2, (i) assessment, (j) intervention, (k) implementation, management, and supervision, and (l) professional and ethical. There was a total of 176 MCQs across the 12 content areas. Each participant took the examination on a desktop or laptop computer. The participants were instructed that they had 4 hr to complete the examination and were not permitted to access their cellphone during the examination or other aspects of the computer other than the examination (e.g., calculator, web browser). One of the researchers proctored the administration of the examination and answered any questions regarding technology (e.g., how to move to the next module) but did not answer any questions about the content of the examination (e.g., the definition of a word).</p> <p>The examination yielded an overall score of correct and incorrect responses. Each participant's performance was converted to a percentage by dividing the total number of correct responses by the total number of questions and multiplying by 100.</p> <hd id="AN0163683943-7">Discrete trial teaching assessment</hd> <p>The implementation of DTT was selected for assessment based on its commonly documented use within ABA-based intervention for autistics/individuals diagnosed with ASD ([<reflink idref="bib13" id="ref30">13</reflink>]). That is, certified behavior analysts working in the autism intervention field will likely be required to implement and train others on the use of DTT. The DTT assessment was a 5-min assessment designed to evaluate the participants' implementation of DTT. This duration was selected based on previous literature reviews ([<reflink idref="bib10" id="ref31">10</reflink>]) and the researchers experience with DTT (i.e., it was deemed a sufficient duration for the participants to display all the skills in the task analysis).</p> <p>All participants implemented DTT with the same child, a 6-year-old male with a diagnosis of ASD. The researchers obtained consent from parents for this child to participate within the study and the researchers monitored to assure that child was not assenting out of any session by vocally indicating they did not want to participate or by engaging in aberrant behavior. These sessions occurred outside of the child's normal behavioral intervention and provided extra opportunities for the child to learn targets which may be helpful in social interaction with the child's peers.</p> <p>Each participant was randomly assigned a different set of three pictures unfamiliar to the child that were provided to the participant for the assessment. The pictures consisted of sport team logos (e.g., St. Louis Cardinals, Los Angeles Rams) with the team names written on the back. Each DTT assessment began with the child diagnosed with ASD in the room. The researcher then brought in the participant to the room and provided the instruction,</p> <p>I want you to use discrete trial teaching to teach these labels receptively [while handing the participant the three pictures with the target names written on the back]. You have five minutes, but if you would like to stop before then please let me know. If you require additional materials let me know.</p> <p>Additional materials commonly requested included token boards, duplicates of the target stimuli, and the child's preferred toys/activities. After the participant was provided with any requested additional materials, a timer was set for 5 min. The assessment ended after the timer elapsed or the participant signaled s/he was done. The researcher did not provide any reinforcement, prompting, or feedback during the 5-min assessment.</p> <p>Participant responding was scored using a task analysis (see Table 2). While watching the video of the assessment, the researchers marked yes, no, or not applicable for each skill/behavior on the checklist. "N/A" was scored in situations in which there was no opportunity for the participant to display that behavior/skill and were not included in the total score. For instance, if the participant opted to use errorless teaching methods and no incorrect responses occurred, the participant could not provide corrective feedback for incorrect responses. The scores were then converted into a percentage by dividing the total number of skills/behaviors marked "yes" by the total number of skills/behaviors marked "yes" and "no" and multiplying by 100.</p> <p>Graph</p> <p>Table 2. Discrete Trial Teaching Task Analysis.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /></colgroup><thead><tr><th align="left">Step</th><th align="center">Behavior</th></tr></thead><tbody><tr><td>1</td><td>Participant provided instructions that were clear on every trial</td></tr><tr><td>2</td><td>Participant provided instructions that matched the student's comprehension level on every trial</td></tr><tr><td>3</td><td>Participant provided an instruction only one time per response on every trial</td></tr><tr><td>4</td><td>Participant did not provide readiness cues (e.g., "look at me")</td></tr><tr><td>5</td><td>Participant established student attending/learning how to learn skills prior to delivering an instruction (e.g., looking at stimuli, scanning, looking at instructor, refraining from self-stimulatory behavior)</td></tr><tr><td>6</td><td>Participant reinforced/praised attending/learning how to learn skills</td></tr><tr><td>7</td><td>Participant placed at least three stimuli in front of the student</td></tr><tr><td>8</td><td>Participant rotated stimulus placement</td></tr><tr><td>9</td><td>Participant provided variations of instructions</td></tr><tr><td>10</td><td>Participant began each trial with an instruction or prompt</td></tr><tr><td>11</td><td>Participant provided up to 5 s for the student to respond following an instruction or prompt</td></tr><tr><td>12</td><td>Participant provided all prompts following or simultaneous with the instruction and before provided the student up to 5 s to respond</td></tr><tr><td>13</td><td>Participant faded prompts systematically across trials</td></tr><tr><td>14</td><td>Participant provided corrective feedback following incorrect responses</td></tr><tr><td>15</td><td>Participant provided praise/reinforcement following correct responses</td></tr></tbody></table> </ephtml> </p> <hd id="AN0163683943-8">Conditioning reinforcement assessment</hd> <p>The conditioning reinforcement assessment was a 5-min assessment evaluating the participants' attempt to condition a neutral item to become more preferred than a child's most-preferred item. All participants implemented the conditioning reinforcement assessment with the same child, a 3-year-old male diagnosed with ASD. The researchers obtained consent from parents for this child to participate within the study and the researchers monitored to assure that child was not assenting out of any session by vocally indicating they did not want to participate or by engaging in aberrant behavior. These sessions occurred outside of the child's normal behavioral intervention and provided extra opportunities to condition extra stimuli to potentially be used as reinforcers.</p> <p>The child's preferred item was a Buzz LightyearTM toy which was identified through caregiver and supervisor report. Each participant was randomly assigned a different neutral item to attempt to condition (e.g., jump rope, jacks, and bouncy balls). This was done to prevent any potential carryover effects from other assessments (e.g., if a participant was successful at conditioning in a previous session, it is likely that item would still be preferred in later sessions). Each conditioning reinforcement assessment began with the child diagnosed with ASD in the room with noncontingent access to his preferred item. Next, the researcher brought in the participant to the room and provided the instruction, "I want you to try to make [item] more preferred than Buzz Lightyear to the best of your ability. You have 5 minutes, but if you would like to stop before then please let me know." A timer was then set for 5 min. The assessment ended after the timer elapsed or the participant signaled they were done. The researcher did not provide any reinforcement, prompting, or feedback during the 5-min assessment.</p> <p>Participant responding was scored using a checklist (see Table 3). While watching the video of the assessment, the researchers marked yes, no, or n/a for each skill/behavior on the checklist. "N/A" was scored in situations in which there was no opportunity to for the participant to display that behavior/skill and were not included in the total score. For instance, if the child did not display favorable affect while interacting with an item, the participant could not interact with the item in the same way. The scores were then converted into a percentage by dividing the total number of skills/behaviors marked "yes" by the total number of skills/behaviors marked "yes" and "no" and multiplying by 100.</p> <p>Graph</p> <p>Table 3. Conditioning Reinforcement Task Analysis.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /></colgroup><thead><tr><th align="left">Step</th><th align="center">Behavior</th></tr></thead><tbody><tr><td>1</td><td>Participant demonstrated the functional way to interact with the item</td></tr><tr><td>2</td><td>Participant demonstrated new/novel ways to interact with the item</td></tr><tr><td>3</td><td>Participant maintained favorable affect throughout the majority of the interaction</td></tr><tr><td>4</td><td>Participant interacted with the item/child differently if the student displayed unfavorable affect</td></tr><tr><td>5</td><td>Participant interacted with item/child the same way if the student displayed favorable affect</td></tr><tr><td>6</td><td>Participant incorporated how the student interacts with preferred items into interactions with new item</td></tr><tr><td>7</td><td>Participant provided a choice between the two items</td></tr></tbody></table> </ephtml> </p> <hd id="AN0163683943-9">General behavioral intervention assessment</hd> <p>The purpose of this assessment was to evaluate the participants' overall implementation of behavioral intervention with a child on their caseload with whom they were familiar in their typical therapy context. Each participant was told to select one child diagnosed with ASD they have provided direct behavioral intervention with previously. The children ranged in age from 2 to 7 years old. The researchers obtained consent from parents for this child to participate within the study, and the researchers monitored to assure that child was not assenting out of any session by vocally indicating they did not want to participate or by engaging in aberrant behavior. Each assessment occurred during the child's regularly scheduled clinical sessions in the area in the clinic the session was scheduled; as the tasks selected could help the children's overall treatment goals. The assessment began when the researcher entered the therapy session and provided the instruction,</p> <p>I want you to run a short session with [child's name]. Take five minutes to look over anything and let me know when you are ready to begin. Once you are ready you will have five minutes to work with [child's name]. You can teach whatever you think is best, however you would like to teach.</p> <p>A timer was then set for 5 min, and the assessment ended after the timer elapsed. The researcher did not provide any reinforcement, prompting, or feedback during the 5-min assessment.</p> <p>The videos from the general behavioral intervention assessment were sent to two behavior analysts. Both behavior analysts had board certification and over 10 years of experience with behavioral intervention for autistics/individuals diagnosed with ASD. Neither of the behavior analysts were associated with the researchers and had no knowledge of the participants or the children in the videos. Both behavior analysts had master's degrees and one was pursuing her doctoral degree. In addition, both were the current directors of separate ABA clinics serving autistics/individuals diagnosed with ASD and were responsible for training their staff on the principles of ABA. The two behavior analysts scored each participant's assessment video using a Likert-type scale that included 12 questions (see Table 4). The behavior analysts selected a number from 1 (e.g., not acceptable, not likely, not interested) to 7 (e.g., highly acceptable, highly likely, highly interested) for each question and were provided room to add any additional information.</p> <p>Graph</p> <p>Table 4. General Behavioral Intervention Assessment Questions.</p> <p> <ephtml> <table><colgroup><col align="left" /><col align="char" char="." /></colgroup><thead><tr><th align="left">Number</th><th align="center">Question</th></tr></thead><tbody><tr><td>1</td><td>I found the interventionists approach to teaching acceptable</td></tr><tr><td>2</td><td>I would likely hire the interventionist to work at my agency</td></tr><tr><td>3</td><td>The interventionist looked interested in the child</td></tr><tr><td>4</td><td>The interventionist appears to have a foundational knowledge in behavior analytic principles and procedures</td></tr><tr><td>5</td><td>The interventionist used behavior analytic procedures and principles</td></tr><tr><td>6</td><td>The interventionist provided an appropriate amount of learning opportunities</td></tr><tr><td>7</td><td>The interventionist provided an appropriate amount of reinforcement</td></tr><tr><td>8</td><td>The interventionist provided an appropriate amount of corrective feedback</td></tr><tr><td>9</td><td>The interventionist was appropriately responsive to the child</td></tr><tr><td>10</td><td>The interventionist targeted skills that seemed appropriate for the child</td></tr><tr><td>11</td><td>The interventionist reinforced attending skills, as appropriate, for the child</td></tr><tr><td>12</td><td>If applicable, the interventionist appropriately handled aberrant behavior</td></tr></tbody></table> </ephtml> </p> <hd id="AN0163683943-10">Data Analysis</hd> <p>A Spearman rank-order correlation coefficient was conducted between MCQ examination scores and performance assessments (i.e., DTT assessment, conditioning reinforcement assessment, and general behavioral intervention assessment). A Spearman rank-order correlation coefficient requires that the relationship between each pair of variables does not change direction ([<reflink idref="bib9" id="ref32">9</reflink>]). This assumption is violated if the points on the scatterplot between any pair of variables appear to shift from a positive to negative or negative to positive relationship. Given the expected variability in scores across the assessments within this study, a Spearman rank-order correlation coefficient was an appropriate test because it does not assume that the variables were normally distributed. Cohen's standard was used to evaluate the strength of the relationships, where coefficients between.10 and.29 represent a small effect size, coefficients between.30 and.49 represent a moderate effect size, and coefficients above.50 indicate a large effect size ([<reflink idref="bib8" id="ref33">8</reflink>]).</p> <hd id="AN0163683943-11">Interobserver Reliability</hd> <p>The primary investigator scored the 25 participants' DTT and conditioning reinforcement assessments. A second independent observer scored 36% of the participants' sessions for the DTT assessment and 36% of sessions for conditioning reinforcement. For both conditions, interobserver agreement (IOA) was calculated by dividing the number of agreements of steps done correctly, incorrectly, or omitted by the number of agreements plus disagreements and multiplying by 100. Agreements were defined as both observers scoring the same response on a step of the task analysis. Disagreements were defined as the observers scoring a different response on a step of the task analysis. Interobserver agreement for discrete teaching probes was 95% (range, 80%–100% per participant) and IOA for conditioning reinforcer probes was 94% (range, 71%–100% per participant).</p> <hd id="AN0163683943-12">Results</hd> <p></p> <hd id="AN0163683943-13">Performance Across Assessments</hd> <p>Figure 1 displays the results for all participants on the MCQ examination and each of the assessments. Across the x-axis are the different assessment categories (i.e., MCQ examination, DTT, conditioning reinforcement, and general behavioral intervention). The primary y-axis represents scores on the MCQ examination, DTT assessment, and the conditioning reinforcement assessment. The secondary y-axis represents scores on the general behavioral intervention assessment. Each participant is represented by an individual data point.</p> <p>Graph: Figure 1. Results for all participants on the MCQ examination and each of the assessments. Note. The secondary y-axis represents scores on the general behavioral intervention assessment. Each closed circle represents one of the 25 participants.DTT = discrete trial teaching.</p> <p>The mean score across the 25 participants on the MCQ examination was 53.1% (range 30%–76%). Evaluations conducted by [<reflink idref="bib5" id="ref34">5</reflink>] indicate that completion of all the Behavior Development Solutions learning modules at 100% correct is highly correlated with first-time pass rates on the BCBA exam. As such, none of the participants met the criteria indicative of being highly likely to pass the BCBA examination.</p> <p>Colloquially, lower scores on the performance-based assessments (i.e., DTT, conditioning reinforcement, and general behavioral intervention) can be viewed as areas in which training may be necessary while higher scores can be viewed as the accurate implementation of all necessary steps of the assessed procedures. The mean score across the 25 participants on the DTT assessment was 93.1% (range, 77%–100%). The most common error on the DTT assessment included not providing reinforcement for important, but nontargeted behaviors (e.g., attending and sitting). The mean score across the 25 participants on the conditioning reinforcement assessment was 78.2% (range, 50%–100%). The most common errors on the conditioning reinforcement assessment included not incorporating how the child interacts with known preferred items into how they were engaging with the neutral item and not providing a choice between the two items. The mean score across the 25 participants on the general behavioral intervention assessment was 5.4 (range, 4–6.4).</p> <hd id="AN0163683943-14">Spearman Rank-Order Correlation Coefficient Analysis</hd> <p>Figure 2 displays the results of the Spearman rank-order correlation coefficient analysis. Each closed circle represents one of the 25 participants. A regression line has been added to assist the interpretation. The x-axis on each of the panels represents the MCQ examination scores. It should be noted that the x-axis ranges from 20 to 80. The y-axis on the top panel represents scores on the DTT assessment and the axis ranges from 75 to 100. Based on Spearman rank-order correlation, the correlation coefficient was 0.307 indicating a weak positive monotonic relationship between MCQ examination scores and DTT assessment scores. That is, as MCQ examination scores increased so did DTT assessment scores. However, when the correlation was examined using Holm corrections to adjust for multiple comparisons based on an alpha value of 0.05 there was no significant correlation (<emph>p</emph> =.134). Thus, indicating scores on the MCQ examination were not indicative of how participants performed on the DTT assessment.</p> <p>Graph: Figure 2. Results of the Spearman rank-order correlation coefficient analysis. Note. Each closed circle represents one of the 25 participants. Regression lines have been added to assist in interpretation.DTT = discrete trial teaching.</p> <p>The y-axis (range, 40–100) on the middle panel represents the scores on the conditioning reinforcement assessment. Based on Spearman rank-order correlation, the correlation coefficient was −0.236 indicating a weak negative monotonic relationship between MCQ examination scores and scores on the conditioning reinforcement assessment. That is, as MCQ examination scores increased, scores on the conditioning reinforcer assessment decreased. However, when the correlation was examined using Holm corrections to adjust for multiple comparisons based on an alpha value of 0.05, there was no significant correlation (<emph>p</emph> =.252). Thus, scores on the MCQ examination were not indicative of how participants performed during the conditioning reinforcement assessment.</p> <p>The y-axis (range, 3.5–7.0) on the bottom panel represents the scores on the general behavioral intervention assessment. Based on the Spearman rank-order correlation, the correlation coefficient was −0.160 indicating a very weak negative monotonic relationship between MCQ examination scores and scores on the general behavior intervention assessment. Meaning as MCQ examination scores increased, the general behavior intervention assessment scores decreased. However, when the correlation was examined using Holm corrections to adjust for multiple comparisons based on an alpha value of 0.05,there was no significant correlation (<emph>p</emph> =.441). Thus, indicating scores on the MCQ examination were not indicative of how participants scored on the general behavioral intervention assessment.</p> <hd id="AN0163683943-15">Discussion</hd> <p>The purpose of this study was to investigate the relationship between performance on an analogue of the BCBA examination and performance of common ABA-based procedures used with autistics/individuals diagnosed with ASD. Specifically, scores on the [<reflink idref="bib5" id="ref35">5</reflink>] pre/post examination testing modules were compared to scores on the assessments of participants' implementation of DTT, conditioning reinforcement, and general behavioral implementation with autistics/individuals diagnosed with ASD. The results indicated that MCQ examination scores were not significantly correlated with any of the performance assessments. Said differently, answers on the MCQ examination were not predictive of accurate performance while conducting DTT, conditioning reinforcers, or general behavioral intervention. These results have several implications for those within the field of behavioral interventions for autistics/individuals diagnosed with ASD.</p> <p>Perhaps the most substantial implication relates to MCQ examination as the final determinative step in obtaining behavior analytic certification. While the use of MCQ examinations in this way aligns with standard practice in the certification process and permits legal defensibility ([<reflink idref="bib7" id="ref36">7</reflink>]), the results of this study provide preliminary data that MCQ examination performance may not be an accurate representation of a candidate's behavioral repertoire with respect to performance of common ABA-based procedures. As such, the results provide some experimental support for [<reflink idref="bib20" id="ref37">20</reflink>] statement that, "certification cannot guarantee effective services, but it can increase the likelihood that the practitioner meets some minimal qualifications" (p. 158).</p> <p>These results are consistent with previous critiques of certification, which have suggested the final determinative step in obtaining behavior analytic certification should involve additional methods of assessing certficants' behavior analytic repertoires, as the final or an additional step on the way to certification (e.g., [<reflink idref="bib12" id="ref38">12</reflink>], [<reflink idref="bib11" id="ref39">11</reflink>]). Others that have noted this concern have suggested assessing candidate performance of task list skills in vivo or via video submission (e.g., [<reflink idref="bib12" id="ref40">12</reflink>]). Doing so would resemble other accredited certifications in other areas of practice (e.g., Autism Diagnostic Observation Schedule; [<reflink idref="bib14" id="ref41">14</reflink>]). Unfortunately, the number of applications for behavior analytic certifications may create potential barriers to making such a change (e.g., cost prohibitive; [<reflink idref="bib7" id="ref42">7</reflink>]; [<reflink idref="bib18" id="ref43">18</reflink>]). Nonetheless, this method of assessment could provide certifying agencies a method to ameliorate concerns of say–do correspondence and, potentially, guarantee certificant performance, which would be an important step in ensuring the protection of consumers of ABA-based procedures and better ensure that autistics/individuals diagnosed with ASD receive the highest quality of treatment possible.</p> <p>The results of this study may also be informative for individuals tasked with hiring and/or training individuals to implement ABA-based procedures in clinic and school settings. One stated purpose of certification is to make "it easier for employers, funders, and consumers, among others, to verify that you have certain knowledge, skills, and abilities and are not under disciplinary sanctions" ([<reflink idref="bib3" id="ref44">3</reflink>], p. 2). Despite this purpose, this study provides preliminary evidence that a certification possessed by the individual hired does not correlate with the skills assessed within this study. Therefore, when hiring an individual holding a credential, an assessment of performance may be helpful to obtain an accurate picture of the individual's behavior analytic repertoire with respect to implementation. That way, agencies and schools are not hiring solely based on a certification but rather based on demonstration of implementing or supervising procedures for autistics/individuals diagnosed with ASD. Thus, possibly improving the quality of intervention from the onset of hiring in efforts to improve outcomes for autistics/individuals diagnosed with ASD.</p> <p>Another implication relates to the way in which the general behavioral intervention was assessed. This assessment differed from the other two assessments (i.e., DTT and conditioning reinforcement) in that it did not measure discrete, operationalized behaviors within a task analysis. Instead, two professionals observed a video of the assessment and answered several questions about the video. This method of assessment could be considered a measure of social validity ([<reflink idref="bib22" id="ref45">22</reflink>]). Consequently, the general behavioral intervention assessment provides methods for future similar studies to evaluate the correlation between MCQ examination scores and consumer perceptions of the interventionists' behavioral intervention repertoires. It should be noted, however, that the brevity with which general behavioral intervention was assessed (i.e., 5 min) may not be a representative sample of the participants' implementation of behavioral intervention. Future studies may find that longer durations may be necessary to provide the most valid evaluation of general behavioral intervention implementation.</p> <p>Despite its implications, this study has several limitations that warrant discussion and should be considered when evaluating the results. First, all participants were recruited via a convenience sample from the same agency. As such, the generality of the results to professionals outside of this agency is severely limited. Future researchers should attempt to replicate these findings with a more diverse group of participants (e.g., different agencies, multiple countries). Second, the methods employed with the MCQ examination were selected in an effort to closely resemble possible outcomes on the BCBA examination (Behavior Development Spoliations, n.d.). The actual BCBA examination is proprietary, which is likely necessary to ensure the possibility of accreditation. Nonetheless, these constraints prevent the possibility of a direct comparison to the BCBA examination. We hope that this study creates the motivation as well as an example of methods for certifying agencies to conduct and publish results of more direct comparisons without sacrificing efforts to ensure accreditation. A third limitation is that none of the participants had a BCBA certification and not all participants would have met the qualifications to become eligible for the BCBA examination (e.g., degree requirements and coursework requirement). This decision was made in an effort to gather a more heterogeneous sample of MCQ examination scores. That is, we predicted that individuals who were ineligible for the BCBA examination would provide scores that differed from individuals who were eligible for the BCBA examination. Nonetheless, it rema possible that correlations could be obtained if all eligibility requirements for the examination were met (e.g., degree, coursework, supervised experience, and application for exam).</p> <p>Fourth, this study only evaluated correlations with respect to three implementation assessments. It is possible that significant correlations could be obtained when evaluating other ABA-based procedures (e.g., escape extinction, shaping). Furthermore, the assessments were researcher designed. It is unlikely that the assessments will represent the desired implementation across all agencies providing ABA-based services for autistics/individuals diagnosed with ASD. While this is certainly a limitation, it is important to note that in the general behavioral intervention assessment, the participants could have implemented any ABA-based procedures deemed appropriate for the child with whom they were working. Nonetheless, this represents a limitation with respect to the generality of the findings. Future researchers could work across agencies and researchers to provide a level of validation for the task lists and then evaluate correlations between each skill on task lists and scores on an MCQ examination. Fifth, in the course of the certification process, the BCBA individuals are required to pass the MCQ examination before being granted certification to practice. In this study, however, all of the participants essentially failed the MCQ examination and would not have been provided certification to practice. As a result, any possible correlation between higher scores on the MCQ examination and performance remains unknown. Nonetheless, scores from the performance assessments indicated many of the participants would have performed well with respect to the skills assessed despite their scores on the MCQ examination.</p> <p>Despite its limitations, this study provides, to the authors' knowledge, the first experimental evaluation of the relationship between certification-level MCQ examination and the application of ABA-based procedures for autistics/individuals diagnosed with ASD. These data should be viewed as preliminary and much more research will be needed evaluating the conditions under which MCQ examination scores are and are not predictive of the application of ABA-based procedures. Ideally, future studies will have the resources to access actual BCBA examination opposed to an analogue with strong correlations between the modules and pass rates on the BCBA examination. Furthermore, based on the results of this study, future studies should include a larger and more diverse population, assessment of more ABA-based procedures, and evaluate if higher scores on MCQ examination, those higher than found in this study, predict superior implementation. It may also be fruitful to evaluate correlations across the various components of certification requirements. For instance, performance during field work may be more likely to be correlated with performance on ABA-based procedures. These future studies will help to build an empirical literature base that can inform changes in the certification process. If the purpose of certification is to ensure the "certificate holder has met the established requirements for certification and, therefore, has at least entry-level behavior analysis competence" ([<reflink idref="bib19" id="ref46">19</reflink>], p. 31), this line of research will be invaluable to help ensure that purpose is met. Ultimately, this line of research will bolster efforts to help improve the lives of autistics/individuals diagnosed with ASD.</p> <p>The authors wish to thank Behavioral Development Solutions for providing us with free access to assessments throughout the study.</p> <ref id="AN0163683943-16"> <title> Footnotes </title> <blist> <bibl id="bib1" idref="ref2" type="bt">1</bibl> <bibtext> The first, second, and third, and fourth authors conceptualized the original paper and wrote the original draft. The fifth and sixth authors made major contributions and changes throughout the process.</bibtext> </blist> <blist> <bibl id="bib2" idref="ref18" type="bt">2</bibl> <bibtext> The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibl id="bib3" idref="ref20" type="bt">3</bibl> <bibtext> The author(s) received no financial support for the research, authorship, and/or publication of this article.</bibtext> </blist> <blist> <bibl id="bib4" idref="ref6" type="bt">4</bibl> <bibtext> All procedures performed in studies involving human participants were in accordance with ethical standards of the institutional research committee and with 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from the parents of all individual participants included in the study.</bibtext> </blist> <blist> <bibl id="bib5" idref="ref5" type="bt">5</bibl> <bibtext> While there are likely a large number of people (e.g., teachers, clinicians, and parents) who have been using ABA-based procedures for decades without certification, data on certified behavior analysts provide us one metric by which we can measure growth within the practice wing of behavior analysis.</bibtext> </blist> <blist> <bibl id="bib6" idref="ref1" type="bt">6</bibl> <bibtext> Access to the pretest/posttest modules from Behavior Development Solutions was provided free of charge for the purposes of this study.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref10" type="bt">7</bibl> <bibtext> * Julia L. Ferguson is also affiliated with Endicott College, Beverly, MA, USA</bibtext> </blist> </ref> <ref id="AN0163683943-17"> <title> References </title> <blist> <bibtext> Ala'i-Rosales S., Zeug N. (2010). Three important things to consider when starting intervention for a child diagnosed with autism. Behavior Analysis in Practice, 3(2), 54–55. https://doi.org/10.1007/BF03391766</bibtext> </blist> <blist> <bibtext> Baer D. M., Wolf M. M., Risley T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97.</bibtext> </blist> <blist> <bibtext> Behavior Analyst Certification Board. (2021). Board certified behavior analyst handbook. https://<ulink href="http://www.bacb.com/wp-content/uploads/2020/11/BCBAHandbook%5f210212.pdf">www.bacb.com/wp-content/uploads/2020/11/BCBAHandbook%5f210212.pdf</ulink></bibtext> </blist> <blist> <bibtext> Behavior Analyst Certification Board. (n.d.). BACB certificant data. https://<ulink href="http://www.bacb.com/BACB-certificant-data">www.bacb.com/BACB-certificant-data</ulink></bibtext> </blist> <blist> <bibtext> Behavior Development Solutions. (n.d.). BCBA/BCaBA products. https://<ulink href="http://www.behaviordevelopmentsolutions.com/bcba-exam-prep">www.behaviordevelopmentsolutions.com/bcba-exam-prep</ulink></bibtext> </blist> <blist> <bibtext> Bibby P., Eikeseth S., Martin N. T., Mudford O. C., Reeves D. (2001). Progress and outcomes for children with autism receiving parent-managed intensive interventions. Research in Developmental Disabilities, 22(6), 425–447. https://doi.org/10.1016/s0891-4222(01)00082-8</bibtext> </blist> <blist> <bibtext> Carr J. E., Nosik M. R., DeLeon I. G. (2017). The Registered Behavior Technician™ credential: A response to Leaf et al. (2017). Behavior Analysis in Practice, 10(2), 164–166. https://doi.org/10.1007/s40617-017-0172-1</bibtext> </blist> <blist> <bibl id="bib8" idref="ref33" type="bt">8</bibl> <bibtext> Cohen J. (1988). Statistical power analysis for the behavioral sciences (Vol. 2). Lawrence Erlbaum.</bibtext> </blist> <blist> <bibl id="bib9" idref="ref32" type="bt">9</bibl> <bibtext> Conover W. J., Iman R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician, 35(3), 124–129. https://doi.org/10.2307/2683975</bibtext> </blist> <blist> <bibtext> Leaf J. B., Aljohani W. A., Milne C. M., Ferguson J. L., Cihon J. H., Oppenheim-Leaf M. L., McEachin J., Leaf R. (2019). Training behavior change agents and parents to implement discrete trial teaching: A literature review. Review Journal of Autism and Developmental Disorders, 6(1), 26–39. https://doi.org/10.1007/s40489-018-0152-6</bibtext> </blist> <blist> <bibtext> Leaf J. B., Leaf R., McEachin J., Bondy A., Cihon J. H., Detrich R., Eshleman J., Ferguson J. L., Foxx R. M., Freeman B. J., Gerhardt P., Glenn S. S., Miller M., Milne C. M., Mountjoy T., Parker T., Pritchard J., Ross R. K., Saunders M. S., Streff T. (2020). The importance of professional discourse for the continual advancement of practice standards: The RBT® as a case in point. Journal of Autism and Developmental Disorders, 51, 1789–1801. https://doi.org/10.1007/s10803-020-04631-z</bibtext> </blist> <blist> <bibtext> Leaf J. B., Leaf R., McEachin J., Taubman M., Smith T., Harris S. L., Freeman B. J., Mountjoy T., Parker T., Streff T., Volkmar F. R., Waks A. (2017). Concerns about the Registered Behavior Technician™ in relation to effective autism intervention. Behavior Analysis in Practice, 10(2), 154–163. https://doi.org/10.1007/s40617-016-0145-9</bibtext> </blist> <blist> <bibtext> Lerman D. C., Valentino A. L., LeBlanc L. A. (2016). Discrete trial training. In Lang R., Hancock T. B., Singh N. N. (Eds.), Early intervention for young children with autism spectrum disorder (pp. 47–83). Springer. https://doi.org/10.1007/978-3-319-30925-5_3</bibtext> </blist> <blist> <bibtext> Lord C., Rutter M., DiLavore P. C., Risi S. (1999). Autism diagnostic observation schedule-WPS. Western Psychological Services.</bibtext> </blist> <blist> <bibtext> Moore J., Shook G. L. (2001). Certification, accreditation, and quality control in behavior analysis. The Behavior Analyst, 24(1), 45–55. https://doi.org/10.1007/BF03392018</bibtext> </blist> <blist> <bibtext> Porto R. B., Oliveira-Castro J. M. (2013). Say-do correspondence in brand choice: Interaction effects of past and current contingencies. The Psychological Record, 63(2), 345–362. https://doi.org/10.11133/j.tpr.2013.63.2.009</bibtext> </blist> <blist> <bibtext> Risley T. R. (1975). Certify procedures not people. In Scott Wood W. (Ed.), Issues in evaluating behavior modification (pp. 159–181). Research Press.</bibtext> </blist> <blist> <bibtext> Shook G. L. (2005). An examination of the integrity and future of the Behavior Analyst Certification Board credentials. Behavior Modification, 29(3), 562–574. https://doi.org/10.1177/0145445504274203</bibtext> </blist> <blist> <bibtext> Shook G. L., Ala'i-Rosales S., Glenn S. S. (2002). Training and certifying behavior analysts. Behavior Modification, 26(1), 27–48. https://doi.org/10.1177/0145445502026001003</bibtext> </blist> <blist> <bibtext> Starin S., Hemingway M., Hartsfield F. (1993). Credentialing behavior analysts and the Florida behavior analysis certification program. The Behavior Analyst Today, 16(2), 153–166. https://doi.org/10.1007/BF03392620</bibtext> </blist> <blist> <bibtext> Weiss M. J., Shook G. L. (2010). Resources on training requirements for applied behavior analysts: The Behavior Analyst Certification Board and the autism special interest group consumer guidelines. European Journal of Behavior Analysis, 11(2), 217–222. https://doi.org/10.1080/15021149.2010.11434345</bibtext> </blist> <blist> <bibtext> Wolf M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11(2), 203–214. https://doi.org/10.1901/jaba.1978.11-203</bibtext> </blist> </ref> <aug> <p>By Justin B. Leaf; Joseph H. Cihon; Julia L. Ferguson; Christine Milne; Ronald Leaf and John McEachin</p> <p>Reported by Author; Author; Author; Author; Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib19" firstref="ref3"></nolink> <nolink nlid="nl2" bibid="bib21" firstref="ref4"></nolink> <nolink nlid="nl3" bibid="bib12" firstref="ref11"></nolink> <nolink nlid="nl4" bibid="bib11" firstref="ref12"></nolink> <nolink nlid="nl5" bibid="bib15" firstref="ref15"></nolink> <nolink nlid="nl6" bibid="bib17" firstref="ref16"></nolink> <nolink nlid="nl7" bibid="bib16" firstref="ref23"></nolink> <nolink nlid="nl8" bibid="bib13" firstref="ref30"></nolink> <nolink nlid="nl9" bibid="bib10" firstref="ref31"></nolink> <nolink nlid="nl10" bibid="bib20" firstref="ref37"></nolink> <nolink nlid="nl11" bibid="bib14" firstref="ref41"></nolink> <nolink nlid="nl12" bibid="bib18" firstref="ref43"></nolink> <nolink nlid="nl13" bibid="bib22" firstref="ref45"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1377943
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluating the Relationship between Performance on a Multiple-Choice Examination and Common ABA-Based Procedures
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Leaf%2C+Justin+B%2E%22">Leaf, Justin B.</searchLink><br /><searchLink fieldCode="AR" term="%22Cihon%2C+Joseph+H%2E%22">Cihon, Joseph H.</searchLink><br /><searchLink fieldCode="AR" term="%22Ferguson%2C+Julia+L%2E%22">Ferguson, Julia L.</searchLink><br /><searchLink fieldCode="AR" term="%22Milne%2C+Christine%22">Milne, Christine</searchLink><br /><searchLink fieldCode="AR" term="%22Leaf%2C+Ronald%22">Leaf, Ronald</searchLink><br /><searchLink fieldCode="AR" term="%22McEachin%2C+John%22">McEachin, John</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Focus+on+Autism+and+Other+Developmental+Disabilities%22"><i>Focus on Autism and Other Developmental Disabilities</i></searchLink>. Jun 2023 38(2):113-123.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications and Hammill Institute on Disabilities. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Applied+Behavior+Analysis%22">Applied Behavior Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Certification%22">Certification</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+Personnel%22">Professional Personnel</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Correlation%22">Correlation</searchLink><br /><searchLink fieldCode="DE" term="%22Autism+Spectrum+Disorders%22">Autism Spectrum Disorders</searchLink><br /><searchLink fieldCode="DE" term="%22Intervention%22">Intervention</searchLink><br /><searchLink fieldCode="DE" term="%22Children%22">Children</searchLink><br /><searchLink fieldCode="DE" term="%22Behavior+Problems%22">Behavior Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Clinics%22">Clinics</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement%22">Reinforcement</searchLink><br /><searchLink fieldCode="DE" term="%22Conditioning%22">Conditioning</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/10883576221110170
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1088-3576<br />1538-4829
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: To ensure the quality of training, skills developed and maintained, and expected standards for behavior analysts, several systems have been put into place across the years. Of these systems, perhaps the most widespread was the development of the Behavior Analyst Certification Board in 1998. Once all requirements are met, the final determinative step in obtaining certification is a multiple-choice examination. The use of a multiple-choice examination as the final determinative step in obtaining behavior analytic certification may create some concerns for behavior analysts. Identification of a positive correlation between the multiple-choice examination and performance may help ameliorate these concerns. The purpose of this study was to evaluate any potential correlations between scores on a popular commercially available prep examination and basic applied behavior analysis (ABA)--based autism intervention procedures with 25 professionals. Overall, the results indicated that multiple-choice examination scores were not significantly correlated with any assessments of performance.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1377943
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1377943
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/10883576221110170
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 113
    Subjects:
      – SubjectFull: Applied Behavior Analysis
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Certification
        Type: general
      – SubjectFull: Professional Personnel
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Correlation
        Type: general
      – SubjectFull: Autism Spectrum Disorders
        Type: general
      – SubjectFull: Intervention
        Type: general
      – SubjectFull: Children
        Type: general
      – SubjectFull: Behavior Problems
        Type: general
      – SubjectFull: Clinics
        Type: general
      – SubjectFull: Reinforcement
        Type: general
      – SubjectFull: Conditioning
        Type: general
    Titles:
      – TitleFull: Evaluating the Relationship between Performance on a Multiple-Choice Examination and Common ABA-Based Procedures
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Leaf, Justin B.
      – PersonEntity:
          Name:
            NameFull: Cihon, Joseph H.
      – PersonEntity:
          Name:
            NameFull: Ferguson, Julia L.
      – PersonEntity:
          Name:
            NameFull: Milne, Christine
      – PersonEntity:
          Name:
            NameFull: Leaf, Ronald
      – PersonEntity:
          Name:
            NameFull: McEachin, John
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 1088-3576
            – Type: issn-electronic
              Value: 1538-4829
          Numbering:
            – Type: volume
              Value: 38
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Focus on Autism and Other Developmental Disabilities
              Type: main
ResultId 1