View in EDS HTML Full Text PDF Full Text

Tailoring Support Following Summative Assessments: A Latent Profile Analysis of Student Outcomes across Five Medical Specialities

Saved in:
Bibliographic Details
Title: Tailoring Support Following Summative Assessments: A Latent Profile Analysis of Student Outcomes across Five Medical Specialities
Language: English
Authors: Huiming Ding (ORCID 0000-0002-5747-7796), Matt Homer (ORCID 0000-0002-1161-5938)
Source: Advances in Health Sciences Education. 2025 30(2):459-473.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 15
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Summative Evaluation, Profiles, Statistical Analysis, Outcomes of Education, Medical Education, Medical Students, Undergraduate Students, Objective Tests, Classification, Test Format, Specialization
DOI: 10.1007/s10459-024-10357-9
ISSN: 1382-4996
1573-1677
Abstract: Summative assessments are often underused for feedback, despite them being rich with data of students' applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students' performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,…). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1484818
Database: ERIC
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwG5XYY0Re2B_D9AEHbE3vesAAAA4jCB3wYJKoZIhvcNAQcGoIHRMIHOAgEAMIHIBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBzFCrYmQu_IOQpIwQIBEICBmvyqVdCgJ4hvR6Fiyx9Sshln8NmVWiGQqPhXc3noo-fp3h9uA9hCQsxJHYXgkEtbmweuMH8kGHuZRz8VfTccOXv3x2yBjnZuHmnxM771Wx7p2McP82bo9HUhw3YgM5fePdabVBPXjBkRK24ta2iieSGU7J7BQw5u8G9qAqMvbL45WZMPmOkGn6GIodEZ7H1kvIEgFEQNh9yhVh4=
Text:
  Availability: 1
  Value: <anid>AN0184191454;oak01apr.25;2025Apr04.02:33;v2.2.500</anid> <title id="AN0184191454-1">Tailoring support following summative assessments: a latent profile analysis of student outcomes across five medical specialities </title> <p>Summative assessments are often underused for feedback, despite them being rich with data of students' applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students' performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,...). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.</p> <p>Keywords: Latent profile analysis; Summative assessment; AKT; OSCE; Specialities; Feedback; Education Specialist Studies In Education</p> <hd id="AN0184191454-2">Introduction</hd> <p>Summative assessment like Applied Knowledge Tests (AKTs) and Objective Structured Clinical Examinations (OSCEs), provide a key opportunity for generating appropriate feedback for students and faculty to facilitate the improvement of learning and curriculum and aid the development of the academic/clinical programme (Lineberry, [<reflink idref="bib27" id="ref1">27</reflink>]; Norcini et al., [<reflink idref="bib40" id="ref2">40</reflink>]; Park et al., [<reflink idref="bib43" id="ref3">43</reflink>]). However, summative data are often underused in this respect, despite the well-known calls to promote the formative use of summative assessment outcomes (Black et al., [<reflink idref="bib6" id="ref4">6</reflink>]; Lineberry, [<reflink idref="bib27" id="ref5">27</reflink>]; Schuwirth & van der Vleuten, [<reflink idref="bib46" id="ref6">46</reflink>]). In practice, focus is often put on the pass-fail decision which is although certainly important (Harrison et al., [<reflink idref="bib21" id="ref7">21</reflink>]) with assessment data commonly aggregated into single scores regarding overall performance, and perhaps also performance in different domains or speciality areas (e.g. Acute and Critical Care, Psychiatry,...) for use in student feedback. It is not always clear to what extent students meaningfully engage with such feedback to develop their knowledge and improve their clinical skills (Hattie & Timperley, [<reflink idref="bib22" id="ref8">22</reflink>]). As summative assessments are usually administered at the end of the academic year, their primary function is often considered by students as evaluation for progression rather than informing learning and development. Feedback after these types of assessments is often under-used by students, especially for those who have passed the exam (Harrison et al., [<reflink idref="bib20" id="ref9">20</reflink>], [<reflink idref="bib21" id="ref10">21</reflink>], [<reflink idref="bib19" id="ref11">19</reflink>]) despite their acknowledged compensatory nature where overall score might mask weak performance in some areas (Cizek & Bunch, [<reflink idref="bib14" id="ref12">14</reflink>]). Medical schools, with limited faculty resources, naturally tend to focus their feedback efforts on those who failed the exam (Harrison et al., [<reflink idref="bib20" id="ref13">20</reflink>]). This implies that passing students might not always have an accurate conception of their competence, particularly in sub-domains. Further, evidence suggests that low performers are those who are more likely to struggle to set appropriate learning goals to address specific weak areas (Chang et al., [<reflink idref="bib12" id="ref14">12</reflink>]). This suggests that schools should better identify the students in need of support in specific areas, provide sufficient learning opportunities, and develop students accordingly (GMC, [<reflink idref="bib16" id="ref15">16</reflink>]).</p> <p>It is therefore important to identify and understand the performance patterns of distinct sub-groups of students so that tailored remediation can be implemented to effectively address the specific weaknesses of such sub-groups, thereby improving future learning. Nevertheless, single scores per se do not directly indicate specific patterns of knowledge and skills for sub-groups (Altshuler et al., [<reflink idref="bib2" id="ref16">2</reflink>]). By contrast, the person-centred methodological approach, latent profile analysis (LPA) (or latent class analysis, LCA)[<reflink idref="bib1" id="ref17">1</reflink>], assumes that groupings of individuals, which are not directly observable in the data, can be revealed <emph>a posteriori</emph> based on the analysis of observed scores or responses (Jung & Wickrama, [<reflink idref="bib26" id="ref18">26</reflink>]; Marsh et al., [<reflink idref="bib31" id="ref19">31</reflink>]). This approach identifies the underlying latent heterogeneity in students (Lubke & Muthén, [<reflink idref="bib29" id="ref20">29</reflink>]) and classifies them into distinct sub-groups who have shared performance characteristics (Weller et al., [<reflink idref="bib55" id="ref21">55</reflink>]) based on the set of measures (scores or sub-scores) used. Compared with a simple pass/fail classification, or aggregate scores or rankings, LPA allows for the identification and understanding of the particular needs of sub-groups of students according to their common characteristics (i.e. relative strengths and weakness). It also provides additional insights that can help deliver adaptive teaching (Boscardin, [<reflink idref="bib8" id="ref22">8</reflink>]; Ma, [<reflink idref="bib32" id="ref23">32</reflink>]) based on scoring patterns identified in the summative examinations.</p> <p>In terms of the limited LPA/LCA literature in medical education, Boscardin ([<reflink idref="bib8" id="ref24">8</reflink>]) profiled students' history taking, physical exam, and physician-patient interaction skills in the Clinical Performance Examination (CPX). Besides identifying two different groups of students for remediation – one group needs support for all the three types of clinical skills across the related scenario cases while the other group needs support particularly for physician-patient interaction skills, her study also suggests that setting a cut-off value for each CPX case instead of an overall cut score is preferred to ensure all the 'pass' students are those having the necessary skills in all the clinical areas (Boscardin, [<reflink idref="bib8" id="ref25">8</reflink>]). More recently, she and colleagues ([<reflink idref="bib9" id="ref26">9</reflink>]) used LPA-type methods to explore student profiles of learning orientation, learning approaches, and perception of the learning environment. In their study, the four identified profile groups were found related to students' clerkship performance (Boscardin et al., [<reflink idref="bib9" id="ref27">9</reflink>]). In another LPA study identifying the latent profiles of medical students' four domains of communication skills demonstrated in the Comprehensive Clinical Skills Exam (CCSE), Altshuler et al. ([<reflink idref="bib2" id="ref28">2</reflink>]) evidenced the typical weaknesses of the low performing groups which led to them failing the exam. Their findings provide detailed guidance on the provision of targeted feedback and personalised education on communication skills. Mak-van der Vossen et al. ([<reflink idref="bib30" id="ref29">30</reflink>]) identified three different profiles of medical students' unprofessional behaviour, which were labelled as (<reflink idref="bib1" id="ref30">1</reflink>) "Poor reliability", (<reflink idref="bib2" id="ref31">2</reflink>) "Poor reliability and poor insight", and (<reflink idref="bib3" id="ref32">3</reflink>) "Poor reliability, poor insight and poor adaptability". LPA has also been used to sort different specialties into distinct groups sharing similar intrinsic characteristics in terms of working environment (e.g. research intensive, procedurally-focused, clinical focused, etc.), which can then be used to inform medical students on their specialty choice (Balas et al., [<reflink idref="bib5" id="ref33">5</reflink>]).</p> <p>Acquisition of basic clinical knowledge and skills across different clinical specialties is essential for medical students to prepare themselves for undertaking medical practice as doctors in various healthcare settings and lays the foundation for their future career choices (GMC, [<reflink idref="bib16" id="ref34">16</reflink>], [<reflink idref="bib17" id="ref35">17</reflink>]; VanOrder et al., [<reflink idref="bib53" id="ref36">53</reflink>]). To our knowledge, few studies have investigated whether there are distinct patterns of students' performance across different clinical specialities, and how those patterns differentiate across distinct groups of students in both applied knowledge tests and OSCE-type assessments simultaneously. According to Zelesniack et al. ([<reflink idref="bib57" id="ref37">57</reflink>]), advanced undergraduate medical students lack proper understanding of specific competencies that are required for different specialties. Some researchers have also suggested that "acute care is an area of concern for UK graduates" in medicine (Tallentire et al., [<reflink idref="bib52" id="ref38">52</reflink>], p.365) due to the lack of exposure and hands-on experience in medical school (Burridge et al., [<reflink idref="bib11" id="ref39">11</reflink>]). Nevertheless, these claims were made based on doctors' perceptions rather than real assessment data.</p> <p>At the test level, contradictory results have been reported on the correlation between OSCE scores and knowledge test scores in some extant studies (e.g. Auewarakul et al., [<reflink idref="bib4" id="ref40">4</reflink>]; Johnson & Reynard, [<reflink idref="bib25" id="ref41">25</reflink>]; Nguyen et al., [<reflink idref="bib39" id="ref42">39</reflink>]) suggesting that different assessment formats do measure different constructs/domains, at least in some contexts. Students' performance in an OSCE may also be influenced by factors such as their communication skills (Attrill et al., [<reflink idref="bib3" id="ref43">3</reflink>]; Groene et al., [<reflink idref="bib18" id="ref44">18</reflink>]), examiner effects (e.g. Chong et al., [<reflink idref="bib13" id="ref45">13</reflink>]; Homer, [<reflink idref="bib23" id="ref46">23</reflink>]), and testing context (Blaskiewicz et al., [<reflink idref="bib7" id="ref47">7</reflink>]). However, there is scarce research discussing the extent to which profiles of what students "know" and "know how" in the knowledge test align with what they can actually "show how" in practice (i.e. clinical and other skills as measured in the OSCE) (Miller, [<reflink idref="bib35" id="ref48">35</reflink>]). More nuanced research is therefore needed that links both AKT and OSCE data, holistically looking into areas of student relative strength and weakness in applied knowledge and clinical skills.</p> <p>In this study, to better inform teaching and tailor student support, we gain useful insights from AKT and OSCE assessment data at the speciality level by using LPA as our main methodological approach. Our research questions are the following.</p> <p></p> <ulist> <item> How do students group according to AKT outcomes at speciality level?</item> <p></p> <item> How do students group according to OSCE outcomes at speciality level?</item> <p></p> <item> How well do these groupings align with each other?</item> <p></p> <item> What insights can all stakeholders gain from this type of profiling?</item> </ulist> <hd id="AN0184191454-3">Methods</hd> <p></p> <hd id="AN0184191454-4">Data and sample</hd> <p>The datasets that we employed in this study consist of the Undergraduate medicine MBChB programme Year 4 students' summative assessment data from the AKT and the OSCE collected at the end of a recent post-covid academic year at University of Leeds, UK. Both assessments are part 1 of a sequential test (Cookson et al., [<reflink idref="bib15" id="ref49">15</reflink>]; Muijtjens et al., [<reflink idref="bib36" id="ref50">36</reflink>]; Pell et al., [<reflink idref="bib44" id="ref51">44</reflink>]). According to UK's General Medical Council (GMC), "medical school curricula must give medical students:... experience in a range of specialties, in different settings, with the diversity of patient groups that they would see when working as a doctor" (GMC, [<reflink idref="bib16" id="ref52">16</reflink>], p.33). In Year 4, learning focuses on developing clinical experience through the five Integrated Course Units (ICUs) of speciality placements in Acute and Critical Care (ACC), Continuing Care and Cancer (CCC), Gynaecology, Obstetrics and Sexual Health (GOSH), Paediatrics, and Psychiatry. Each ICU takes six weeks. In addition, three days of Integrated "Theme" teaching across the year provide students with opportunities to integrate their knowledge across different specialities. By the end of the academic year, students are required to have acquired all mandatory skills of these specialities, which prepare them to be a safe foundation year doctor, build a basis for their future career choices, and enhance their capabilities of working effectively in interprofessional teams. Students' knowledge and skills in the same five specialities are assessed across both the summative AKT and OSCE. The size of the Year 4 cohort is 295. The University of Leeds gave permission for this anonymized data to be used for research. The Chair of the University of Leeds School of Medicine research ethics committee and the Head of Assessment at the School confirmed to the authors that formal ethics approval for this study was not required as it involved the use of routinely collected student assessment data which were fully anonymized prior to analysis.</p> <p>The AKT (Cronbach's alpha = 0.82) consisted of 147 multiple-choice items covering the five specialities with around 30 item each, all scored 0/1. The OSCE (omega.total = 0.78) consisted of 14 stations, assessing students' clinical skills relevant to the five specialities (and also communication skills and other generic skills). Each specialty was the focus in two stations, and the other four stations had more integrated scenarios in terms of covering content tasks across the specialities. Students' percent correct scores in specialities in the AKT and in the OSCE were calculated and used as the indicator variables for LPA modelling. Five such indicators by speciality were constructed based on the AKT dataset. Similarly, five indicators, one for each speciality, and one indicator for integrated specialities, were created based on the OSCE checklist scores provided by examiners.</p> <p>Table 1 summarises the descriptive statistics of students' performance for each speciality indicator. It also includes the speciality-level 'percent pass marks' based on the relevant Ebel ratings (McKinley & Norcini, [<reflink idref="bib33" id="ref53">33</reflink>]) for AKT, and borderline regression (Wood et al., [<reflink idref="bib56" id="ref54">56</reflink>]) for OSCE.</p> <p>Table 1 Descriptive statistics of the AKT and OSCE, by specialities</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2" /><th align="left" rowspan="2"><p>Variables</p></th><th align="left" rowspan="2"><p>N of items/stations</p></th><th align="left" rowspan="2"><p>% pass marks</p></th><th align="left" colspan="4"><p>% correct scores</p></th></tr><tr><th align="left"><p>Mean</p></th><th align="left"><p>SD</p></th><th align="left"><p>Min</p></th><th align="left"><p>Max</p></th></tr></thead><tbody><tr><td align="left" rowspan="5"><p>AKT</p></td><td align="left"><p>AKT_ACC</p></td><td align="left"><p>30</p></td><td align="left"><p>54.70</p></td><td align="left"><p>60.24</p></td><td align="left"><p>11.94</p></td><td align="left"><p>23.33</p></td><td align="left"><p>90.00</p></td></tr><tr><td align="left"><p>AKT_CCC</p></td><td align="left"><p>30</p></td><td align="left"><p>50.93</p></td><td align="left"><p>61.41</p></td><td align="left"><p>9.50</p></td><td align="left"><p>26.67</p></td><td align="left"><p>86.67</p></td></tr><tr><td align="left"><p>AKT_GOSH</p></td><td align="left"><p>28</p></td><td align="left"><p>53.11</p></td><td align="left"><p>73.69</p></td><td align="left"><p>9.77</p></td><td align="left"><p>42.86</p></td><td align="left"><p>96.43</p></td></tr><tr><td align="left"><p>AKT_Paediatrics</p></td><td align="left"><p>30</p></td><td align="left"><p>46.60</p></td><td align="left"><p>64.33</p></td><td align="left"><p>10.76</p></td><td align="left"><p>26.67</p></td><td align="left"><p>86.67</p></td></tr><tr><td align="left"><p>AKT_Psychiatry</p></td><td align="left"><p>29</p></td><td align="left"><p>44.38</p></td><td align="left"><p>72.45</p></td><td align="left"><p>10.39</p></td><td align="left"><p>31.03</p></td><td align="left"><p>93.10</p></td></tr><tr><td align="left" rowspan="6"><p>OSCE</p></td><td align="left"><p>OSCE_ACC</p></td><td align="left"><p>2</p></td><td align="left"><p>39.29</p></td><td align="left"><p>49.33</p></td><td align="left"><p>13.71</p></td><td align="left"><p>14.29</p></td><td align="left"><p>80.36</p></td></tr><tr><td align="left"><p>OSCE_CCC</p></td><td align="left"><p>2</p></td><td align="left"><p>53.85</p></td><td align="left"><p>60.61</p></td><td align="left"><p>11.41</p></td><td align="left"><p>16.07</p></td><td align="left"><p>87.50</p></td></tr><tr><td align="left"><p>OSCE_GOSH</p></td><td align="left"><p>2</p></td><td align="left"><p>51.79</p></td><td align="left"><p>61.54</p></td><td align="left"><p>11.24</p></td><td align="left"><p>25.00</p></td><td align="left"><p>91.07</p></td></tr><tr><td align="left"><p>OSCE_Paediatrics</p></td><td align="left"><p>2</p></td><td align="left"><p>30.91</p></td><td align="left"><p>45.28</p></td><td align="left"><p>15.01</p></td><td align="left"><p>10.91</p></td><td align="left"><p>85.45</p></td></tr><tr><td align="left"><p>OSCE_Psychiatry</p></td><td align="left"><p>2</p></td><td align="left"><p>50.00</p></td><td align="left"><p>64.32</p></td><td align="left"><p>11.42</p></td><td align="left"><p>31.03</p></td><td align="left"><p>91.38</p></td></tr><tr><td align="left"><p>OSCE_ICUint</p></td><td align="left"><p>4</p></td><td align="left"><p>51.89</p></td><td align="left"><p>63.41</p></td><td align="left"><p>9.47</p></td><td align="left"><p>35.85</p></td><td align="left"><p>87.74</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0184191454-5">Data analyses</hd> <p></p> <hd id="AN0184191454-6">Latent profile analysis</hd> <p>Data analyses were conducted in Stata 18 (StataCorp, [<reflink idref="bib51" id="ref55">51</reflink>]). We first ran latent profile analysis to identify different groups of students, using students' percent correct scores on the speciality variables. This was done separately for the AKT and OSCE data. There are various fit indices to evaluate the goodness of fit of LPA model solutions statistically and inform a decision on the number of profiles (i.e. student groups). The indices Akaike's information criterion (AIC) (Akaike, [<reflink idref="bib1" id="ref56">1</reflink>]) and Schwarz's Bayesian information criterion (BIC) (Schwarz, [<reflink idref="bib47" id="ref57">47</reflink>]), which are available in Stata 18, were employed in this study. Lower values of AIC and BIC indicate better model fit. BIC is also considered by some researchers as the most reliable indicator (Nylund et al., [<reflink idref="bib41" id="ref58">41</reflink>]; Vermunt, [<reflink idref="bib54" id="ref59">54</reflink>]) and indicates parsimony as well (Weller et al., [<reflink idref="bib55" id="ref60">55</reflink>]). In the case where the lowest BIC and the lowest AIC are observed in different models, the "elbow points" of the changes of these indicator values can be considered to determine the optimal model (Nylund-Gibson & Choi, [<reflink idref="bib41" id="ref61">41</reflink>]; Petras & Masyn, [<reflink idref="bib45" id="ref62">45</reflink>]).</p> <p>With the one-profile solution as the default, we conducted two–profile to six–profile solutions based on the AKT data and the OSCE data respectively, and then identified the optimal model for each dataset. We also examined entropy (Medeiros, [<reflink idref="bib34" id="ref63">34</reflink>]) and the average posterior probability to evaluate the accuracy of profile classification. Entropy values range from 0 to 1, with higher values indicating stronger classification certainty (Jung & Wickrama, [<reflink idref="bib26" id="ref64">26</reflink>]). It is suggested that an entropy value above 0.80 is preferred, and a value between 0.60 and 0.80 is considered acceptable (Muthén, [<reflink idref="bib37" id="ref65">37</reflink>]; Weller et al., [<reflink idref="bib55" id="ref66">55</reflink>]). The average posterior probabilities indicate the average probability at which students are accurately classified into a profile (Muthén & Muthén, [<reflink idref="bib38" id="ref67">38</reflink>]). As suggested by Weller et al. ([<reflink idref="bib55" id="ref68">55</reflink>]), a cut-off value greater than 0.80 is acceptable and higher than 0.90 is ideal for ensuring classification accuracy. Additionally, group size was also considered, following the suggestion that the profile with less than 1% of the student sample should be rejected (Spurk et al., [<reflink idref="bib50" id="ref69">50</reflink>]).</p> <hd id="AN0184191454-7">Comparing profiles with ANOVA</hd> <p>With the profiles determined based on the statistical information criteria, conceptual interpretability and classification size, we used analysis of variance (ANOVA) to compare the mean AKT performance of students of different profile groups in the specialities, and compared these profiles with the overall pass/fail groups as classified from the Ebel standard setting. We also identified the relative strengths and weaknesses for each AKT profile group based on their average scores in each ICU. Similarly, we conducted the equivalent analyses with student profiles identified from OSCE data.</p> <hd id="AN0184191454-8">The relationship between AKT and OSCE profiles</hd> <p>To examine the extent to which AKT and OSCE profiles aligned with each other, we related the AKT data with the OSCE data. To be specific, we coded the profiles ordinally as 1, 2, 3,... according to the overall AKT/OSCE performance levels of the identified groups, and analysed the extent to which student AKT profiles predicted their OSCE profiles using ordinal regression modelling (Long & Freese, [<reflink idref="bib28" id="ref70">28</reflink>]).</p> <hd id="AN0184191454-9">Results</hd> <p></p> <hd id="AN0184191454-10">The AKT profiles</hd> <p>The results of LPA on the AKT data suggest the 4-profile model is the optimal solution, as it has the lowest AIC and BIC values (see Table 2, shaded row).</p> <p>Table 2 LPA statistics of student performance in the AKT</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Model</p></th><th align="left"><p>ll(model)</p></th><th align="left"><p>df</p></th><th align="left"><p>AIC</p></th><th align="left"><p>BIC</p></th><th align="left"><p>Entropy</p></th></tr></thead><tbody><tr><td align="left"><p>2 – profile</p></td><td align="left"><p>-5376.98</p></td><td align="left"><p>16</p></td><td align="left"><p>10785.95</p></td><td align="left"><p>10844.94</p></td><td align="left"><p>0.80</p></td></tr><tr><td align="left"><p>3 – profile</p></td><td align="left"><p>-5332.65</p></td><td align="left"><p>22</p></td><td align="left"><p>10709.29</p></td><td align="left"><p>10790.4</p></td><td align="left"><p>0.75</p></td></tr><tr><td align="left"><p>4 – profile</p></td><td align="left"><p>-5308.28</p></td><td align="left"><p>28</p></td><td align="left"><p>10672.57</p></td><td align="left"><p>10775.8</p></td><td align="left"><p>0.73</p></td></tr><tr><td align="left"><p>5 – profile</p></td><td align="left"><p>-5305.67</p></td><td align="left"><p>34</p></td><td align="left"><p>10679.34</p></td><td align="left"><p>10804.7</p></td><td align="left"><p>0.67</p></td></tr><tr><td align="left"><p>6 – profile</p></td><td align="left"><p>-5298.53</p></td><td align="left"><p>40</p></td><td align="left"><p>10677.07</p></td><td align="left"><p>10824.55</p></td><td align="left"><p>0.76</p></td></tr></tbody></table> </ephtml> </p> <p>In this model, the entropy value 0.73 is also acceptable according to the usual guidelines. The average posterior probabilities of the classification of the profile membership range from 0.82 to 0.97, which are all above the recommended cut-off of 0.80. Four distinct student profiles corresponding to the characteristics related to their achievements in the five specialities in AKT were therefore identified. The four profiles AKTgroup1, AKTgroup2, AKTgroup3, and AKTgroup4 share 3.4%, 23.7%, 41.4% and 31.5% respectively of the total student sample (see Table 3).</p> <p>The student groups identified based on the four-profile solution also generally resonate with the pass/fail[<reflink idref="bib2" id="ref71">2</reflink>] classification of students based on Ebel standard setting. As Table 3 shows, all AKTgroup1 students clearly failed the AKT Sequence 1 exam; all the AKTgroup3 and AKTgroup4 students are those clearly passed the exam; and AKTgroup2 includes the group of students whose performance was relatively close to the borderline. This data further supports the meaningful interpretation of the four-profile solution.</p> <p>Table 3 Statistics of the groups in the 4-profile solution</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>AKT</p><p>Profile</p></th><th align="left" colspan="3"><p>Sample <italic>n</italic> (%)</p></th><th align="left" colspan="4"><p>Average probabilities for most likely latent profile membership</p></th></tr><tr><th align="left"><p>Pass</p></th><th align="left"><p>Fail</p></th><th align="left"><p>Total</p></th><th align="left"><p>1</p></th><th align="left"><p>2</p></th><th align="left"><p>3</p></th><th align="left"><p>4</p></th></tr></thead><tbody><tr><td align="left"><p>AKTgroup1</p></td><td align="left"><p>0 (0.00%)</p></td><td align="left"><p>10 (3.39%)</p></td><td align="left"><p>10 (3.39%)</p></td><td align="left"><p><bold>0.97</bold></p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.03</p></td></tr><tr><td align="left"><p>AKTgroup2</p></td><td align="left"><p>41 (13.90%)</p></td><td align="left"><p>29 (9.83%)</p></td><td align="left"><p>70 (23.73%)</p></td><td align="left"><p>0.01</p></td><td align="left"><p><bold>0.86</bold></p></td><td align="left"><p>0.13</p></td><td align="left"><p>0.00</p></td></tr><tr><td align="left"><p>AKTgroup3</p></td><td align="left"><p>122 (41.36%)</p></td><td align="left"><p>0 (0.00%)</p></td><td align="left"><p>122 (41.36%)</p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.11</p></td><td align="left"><p><bold>0.82</bold></p></td><td align="left"><p>0.07</p></td></tr><tr><td align="left"><p>AKTgroup4</p></td><td align="left"><p>93 (31.53%)</p></td><td align="left"><p>0 (0.00%)</p></td><td align="left"><p>93 (31.53%)</p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.17</p></td><td align="left"><p><bold>0.83</bold></p></td></tr></tbody></table> </ephtml> </p> <p>The mean achievements of these four groups of students in the five specialities in the AKT are displayed in Table 4. One-way ANOVA post hoc tests suggest that the between-group differences in the five specialities are mostly statistically significant with moderate to large effect sizes (<emph>r</emph><sups><emph>2</emph></sups> ranged from 0.40 to 0.60), except for the difference between AKTgroup1 and AKTgroup2 in ACC (<emph>p</emph> = 0.2).</p> <p>Table 4 Mean differences between the AKT profiles in the specialities</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Speciality</p></th><th align="left"><p>AKTgroup1</p><p>M (SD)</p></th><th align="left"><p>AKTgroup2</p><p>M (SD)</p></th><th align="left"><p>AKTgroup3</p><p>M (SD)</p></th><th align="left"><p>AKTgroup4</p><p>M (SD)</p></th><th align="left"><p>F (3, 291)</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic><sup>2</sup></p></th></tr></thead><tbody><tr><td align="left"><p>ACC</p></td><td align="left"><p>43.67 (10.71)</p></td><td align="left"><p>49.62 (8.93)</p></td><td align="left"><p>59.26 (8.00)</p></td><td align="left"><p>71.29 (7.72)</p></td><td align="left"><p>108.52</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.53</p></td></tr><tr><td align="left"><p>CCC</p></td><td align="left"><p>41.33 (10.21)</p></td><td align="left"><p>54.76 (7.69)</p></td><td align="left"><p>61.12 (6.70)</p></td><td align="left"><p>68.96 (6.27)</p></td><td align="left"><p>85.71</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.47</p></td></tr><tr><td align="left"><p>GOSH</p></td><td align="left"><p>52.50 (5.34)</p></td><td align="left"><p>64.44 (6.30)</p></td><td align="left"><p>75.00 (6.48)</p></td><td align="left"><p>81.22 (6.62)</p></td><td align="left"><p>127.97</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.57</p></td></tr><tr><td align="left"><p>Paediatrics</p></td><td align="left"><p>41.33 (8.92)</p></td><td align="left"><p>54.29 (7.7)</p></td><td align="left"><p>64.95 (6.61)</p></td><td align="left"><p>73.55 (6.31)</p></td><td align="left"><p>143.06</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.60</p></td></tr><tr><td align="left"><p>Psychiatry</p></td><td align="left"><p>47.59 (12.36)</p></td><td align="left"><p>66.45 (8.96)</p></td><td align="left"><p>72.87 (8.13)</p></td><td align="left"><p>79.09 (6.63)</p></td><td align="left"><p>65.64</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.40</p></td></tr></tbody></table> </ephtml> </p> <p>Figure 1 graphically illustrates the distribution of the speciality-level AKT performance of the four groups. On average, the lowest achievers AKTgroup1 consistently failed to correctly answer half of the items on four of the specialities, and almost all of them failed to achieve the specialty-level 'pass marks' on the ACC, CCC, and Paediatrics items. AKTgroup2 had more mixed average scores across the specialities, doing well in GOSH and Psychiatry but failing to pass ACC. AKTgroup3, compared with AKTgroup1 and AKTgroup2, had higher average scores; However, some students in this group obtained a score lower than 60% or even 50% in some of the speciality areas (especially ACC and CCC). For the highest-achiever group AKTgroup4, all the students achieved well in all the specialities.</p> <p>Graph: Fig. 1 Distribution of the AKT performance of the four-profile solution</p> <p>In summary, we conclude that the LPA using the AKT outcomes has worked well based on the usual diagnostics/model fit criteria (e.g. Akaike, [<reflink idref="bib1" id="ref72">1</reflink>]; Muthén, [<reflink idref="bib37" id="ref73">37</reflink>]; Schwarz, [<reflink idref="bib47" id="ref74">47</reflink>]; Weller et al., [<reflink idref="bib55" id="ref75">55</reflink>]), producing four groups of students with successively improving performance across the five specialties.</p> <hd id="AN0184191454-11">The OSCE profiles</hd> <p>As to LPA results of students' OSCE data, Table 5 shows that BIC values are similar in the 2-profile and 3-profile models and increased in the other models. AIC values dropped dramatically from 2-profile model to 3-profile model. These data suggest that the 3-profile model is preferred (shaded). This model also has the entropy value (0.82) much higher than that in other models. Moreover, it also had acceptable average posterior probabilities, which range from 0.84 to 0.92 (see Table 6).</p> <p>Table 5 LPA statistics of student performance in the OSCE ICU stations</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Model</p></th><th align="left"><p>ll(model)</p></th><th align="left"><p>df</p></th><th align="left"><p>AIC</p></th><th align="left"><p>BIC</p></th><th align="left"><p>Entropy</p></th></tr></thead><tbody><tr><td align="left"><p>2 – profile</p></td><td align="left"><p>-6767.96</p></td><td align="left"><p>19</p></td><td align="left"><p>13573.92</p></td><td align="left"><p>13643.98</p></td><td align="left"><p>0.72</p></td></tr><tr><td align="left"><p>3 – profile</p></td><td align="left"><p>-6748.8</p></td><td align="left"><p>26</p></td><td align="left"><p>13549.6</p></td><td align="left"><p>13645.46</p></td><td align="left"><p>0.82</p></td></tr><tr><td align="left"><p>4 – profile</p></td><td align="left"><p>-6739.71</p></td><td align="left"><p>33</p></td><td align="left"><p>13545.42</p></td><td align="left"><p>13667.09</p></td><td align="left"><p>0.67</p></td></tr><tr><td align="left"><p>5 – profile</p></td><td align="left"><p>-6734.06</p></td><td align="left"><p>40</p></td><td align="left"><p>13548.12</p></td><td align="left"><p>13695.6</p></td><td align="left"><p>0.66</p></td></tr><tr><td align="left"><p>6 – profile</p></td><td align="left"><p>-6731.89</p></td><td align="left"><p>47</p></td><td align="left"><p>13557.79</p></td><td align="left"><p>13731.08</p></td><td align="left"><p>0.66</p></td></tr></tbody></table> </ephtml> </p> <p>From Table 6, we can see that OSCEgroup1 students all failed the OSCE (standard setting based on borderline regression); about 55% of students grouped into OSCEgroup2 passed the OSCE; and all OSCEgroup3 students clearly passed the OSCE. The three profiles OSCEgroup1, OSCEgroup2, and OSCEgroup3 made up 1.7%, 46.1%, and 52.2% of the student sample respectively.</p> <p>Table 6 Statistics of the groups in the 3-profile solution</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>OSCE</p><p>Profile</p></th><th align="left" colspan="3"><p>Sample <italic>n</italic> (%)</p></th><th align="left" colspan="3"><p>Average probabilities for most likely latent profile membership</p></th></tr><tr><th align="left" /><th align="left"><p>Pass</p></th><th align="left"><p>Fail</p></th><th align="left"><p>Total</p></th><th align="left"><p>1</p></th><th align="left"><p>2</p></th><th align="left"><p>3</p></th></tr></thead><tbody><tr><td align="left"><p>OSCEgroup1</p></td><td align="left"><p>0 (0.00%)</p></td><td align="left"><p>5 (1.69%)</p></td><td align="left"><p>5 (1.69%)</p></td><td align="left"><p><bold>0.84</bold></p></td><td align="left"><p>0.16</p></td><td align="left"><p>0.00</p></td></tr><tr><td align="left"><p>OSCEgroup2</p></td><td align="left"><p>75 (25.42%)</p></td><td align="left"><p>61 (20.68%)</p></td><td align="left"><p>136 (46.10%)</p></td><td align="left"><p>0.00</p></td><td align="left"><p><bold>0.91</bold></p></td><td align="left"><p>0.09</p></td></tr><tr><td align="left"><p>OSCEgroup3</p></td><td align="left"><p>154 (52.20%)</p></td><td align="left"><p>0 (0.00%)</p></td><td align="left"><p>154 (52.20%)</p></td><td align="left"><p>0.00</p></td><td align="left"><p>0.08</p></td><td align="left"><p><bold>0.92</bold></p></td></tr></tbody></table> </ephtml> </p> <p>As shown in Table 7, students in OSCEgroup1 had the lowest mean performance across the specialities in the OSCE; OSCEgroup2 was the group of moderate performers; and OSCEgroup3 had the highest levels of performance. According to one-way ANOVA post hoc tests, the overall between-group differences are statistically significant with small or moderate effect sizes (<emph>r</emph><sups><emph>2</emph></sups> ranged from 0.23 to 0.44), except for the differences between OSCEgroup 1 and OSCEgroup2 in Paediatrics stations and in ICU-integrated stations (<emph>p</emph><subs><emph>Paediatrics</emph></subs> = 0.8, <emph>p</emph><subs><emph>ICUint</emph></subs> = 0.2).</p> <p>Table 7 Mean differences between the OSCE profiles in the specialities</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left"><p>Speciality</p></th><th align="left"><p>OSCEgroup1</p><p>M (SD)</p></th><th align="left"><p>OSCEgroup2</p><p>M (SD)</p></th><th align="left"><p>OSCEgroup3</p><p>M (SD)</p></th><th align="left"><p>F (2, 292)</p></th><th align="left"><p><italic>p</italic></p></th><th align="left"><p><italic>r</italic><sup>2</sup></p></th></tr></thead><tbody><tr><td align="left"><p>ACC</p></td><td align="left"><p>25.71 (6.51)</p></td><td align="left"><p>42.36 (11.13)</p></td><td align="left"><p>56.26 (11.88)</p></td><td align="left"><p>63.73</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.30</p></td></tr><tr><td align="left"><p>CCC</p></td><td align="left"><p>28.21 (8.22)</p></td><td align="left"><p>54.84 (9.39)</p></td><td align="left"><p>66.76 (8.30)</p></td><td align="left"><p>100.21</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.41</p></td></tr><tr><td align="left"><p>GOSH</p></td><td align="left"><p>37.86 (10.96)</p></td><td align="left"><p>54.66 (8.67)</p></td><td align="left"><p>68.39 (8.11)</p></td><td align="left"><p>116.25</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.44</p></td></tr><tr><td align="left"><p>Paediatrics</p></td><td align="left"><p>31.27 (14.17)</p></td><td align="left"><p>37.94 (12.06)</p></td><td align="left"><p>52.21 (14.04)</p></td><td align="left"><p>45.31</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.24</p></td></tr><tr><td align="left"><p>Psychiatry</p></td><td align="left"><p>47.93 (11.07)</p></td><td align="left"><p>59.14 (10.08)</p></td><td align="left"><p>69.42 (9.95)</p></td><td align="left"><p>44.81</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.23</p></td></tr><tr><td align="left"><p>Integrated</p></td><td align="left"><p>51.89 (9.71)</p></td><td align="left"><p>58.63 (8.36)</p></td><td align="left"><p>68.00 (7.89)</p></td><td align="left"><p>52.97</p></td><td align="left"><p>< 0.001</p></td><td align="left"><p>0.27</p></td></tr></tbody></table> </ephtml> </p> <p>Figure 2 displays the OSCE performance distribution across the six predictor variables for the three groups. In general, OSCEgroup 1 had scores lower than 40% in the stations of ACC, CCC, GOSH and Paediatrics and almost all students in this group failed these stations. However, they mostly passed the ICU integrated stations. Compared with OSCEgroup1, OSCEgroup2 had higher and more consistent performance across CCC, GOSH, Psychiatry and integrated stations. Most students in this group passed Psychiatry and integrated stations. Their relatively low performance in other stations meant that 45% of this group failed the OSCE. For OSCEgroup3, students generally performed well across the stations, although had relatively low scores in ACC and Paediatrics.</p> <p>Graph: Fig. 2 Distribution of OSCE performance of the three-profile solution</p> <p>In summary, the LPA using the OSCE outcomes has also worked well based on the usual diagnostics/model fit criteria, this time producing three groups of students with successively improving performance across the specialties and integrated stations.</p> <hd id="AN0184191454-12">The relationships between AKT and OSCE profiles</hd> <p>Comparing the AKT and OSCE profiles, we find that they aligned with each other to an extent (Spearman's rho = 0.45, <emph>p</emph> < 0.001, a moderate, but statistically significant, association). Through running an ordinal regression model, we found that the AKT profile variable significantly and positively predicted OSCE profiles (<emph>p</emph> < 0.001, pseudo R<sups>2</sups> = 0.15).</p> <p>As shown in Table 8, a crosstab of the distribution of students' AKT and OSCE profiles, most of AKTgroup1 and AKTgroup2 students belonged to OSCEgroup1 and OSCEgroup2. Most of AKTgroup3 and AKTgroup4 students were in OSCEgroup3, and no students in these two high AKT profiles were in OSCEgroup1. However, it is noteworthy that there were some AKTgroup1 and AKTgroup2 students demonstrating very good skills in OSCE (i.e. grouped into OSCEgroup3). Also, a number of AKTgroup4 students were in OSCEgroup2, indicating that they did not perform as well in the OSCE as they did in the AKT.</p> <p>Table 8 Cross tabulation of students' AKT and OSCE profiles</p> <p> <ephtml> <table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="2"><p>AKT profiles</p></th><th align="left" colspan="3"><p>OSCE profiles</p></th><th align="left" /></tr><tr><th align="left"><p>OSCEgroup1</p></th><th align="left"><p>OSCEgroup2</p></th><th align="left"><p>OSCEgroup3</p></th><th align="left"><p>Total</p></th></tr></thead><tbody><tr><td align="left"><p>AKTgroup1</p></td><td align="left"><p>2</p></td><td align="left"><p>7</p></td><td align="left"><p>1</p></td><td align="left"><p>10</p></td></tr><tr><td align="left"><p>AKTgroup2</p></td><td align="left"><p>3</p></td><td align="left"><p>51</p></td><td align="left"><p>16</p></td><td align="left"><p>70</p></td></tr><tr><td align="left"><p>AKTgroup3</p></td><td align="left"><p>0</p></td><td align="left"><p>58</p></td><td align="left"><p>64</p></td><td align="left"><p>122</p></td></tr><tr><td align="left"><p>AKTgroup4</p></td><td align="left"><p>0</p></td><td align="left"><p>20</p></td><td align="left"><p>73</p></td><td align="left"><p>93</p></td></tr><tr><td align="left"><p>Total</p></td><td align="left"><p>5</p></td><td align="left"><p>136</p></td><td align="left"><p>154</p></td><td align="left"><p>295</p></td></tr></tbody></table> </ephtml> </p> <hd id="AN0184191454-13">Discussion</hd> <p>Using latent profile analysis of summative assessment data, this study identified four distinct groups of students according to performance patterns in the AKT, and three such groups in the OSCE. Such identification is more fine-grained than single-score based pass/fail classifications, in terms of improving feedback for tailored remediation purpose. According to the usual methodological guidelines (Nylund et al., [<reflink idref="bib41" id="ref76">41</reflink>]; Spurk et al., [<reflink idref="bib50" id="ref77">50</reflink>]; Weller et al., [<reflink idref="bib55" id="ref78">55</reflink>]), the LPA worked well suggesting that these distinct groupings provide meaningful and substantive information about the performance of sub-cohorts of students. For each identified group, we also investigated relative strengths and weaknesses across the five specialities used as indicators in the analysis. The relationship between students' AKT profiles and their OSCE profiles generally showed a positive correlation between the two performances, but did reveal some important variation from this for some individual students.</p> <hd id="AN0184191454-14">Areas for feedback and development</hd> <p>In terms of the practical use of the findings, these classifications can be used to provide more meaningful and effective feedback and to highlight to faculty the specific areas that the lower performing groups need additional support for where appropriate at their later learning phase (e.g. Year 4 resit, or Year 5). The AKT results identify specific learning needs for all four student groups, but particularly for the two lower performing ones who need remediation. For clinical tutors and placement supervisors, the analysis can be used to help them to work collectively to support student learning with tailored strategies and placement management for different groups of students. For example, this include providing AKTgroup1 with additional learning support across all the speciality areas, but especially in ACC, CCC, and Paediatrics, and providing more support for AKTgroup2 in ACC (most of whom were close to or below borderline in this speciality).</p> <p>There is also, perhaps, some important messages in the analysis for elements of the teaching and curriculum – students appear to not have performed as well, relatively, in ACC, compared to the other specialties. This finding supports extant studies which argue that acute care is perceived to be a concerned area for improvement in medical student learning (Burridge et al., [<reflink idref="bib11" id="ref79">11</reflink>]; Tallentire et al., [<reflink idref="bib52" id="ref80">52</reflink>]). According to the year 4 curriculum at School of Medicine at Leeds, amongst the five specialities, ACC is skill-intensive and also the only one incorporating four sub-specialities (i.e. acute medicine, emergency medicine, critical care and anaesthesia), which suggests that it might be more demanding and challenging for students. Future curriculum reviews of ACC might consider wider issues of training, student absence, professionalism and so on help ensure students' sufficient exposure to all the requisite skills, in order to improve performance in ACC (Burridge et al., [<reflink idref="bib11" id="ref81">11</reflink>]; Burford et al., [<reflink idref="bib10" id="ref82">10</reflink>]). As displayed in Table 1; Fig. 1, students, regardless of their AKT profiles, had relatively low scores in ACC and CCC, and high scores in Psychiatry; however, the pass marks set based on Ebel ratings for these ACC and CCC were relatively high, and for Psychiatry was lowest. These data can be used to inform reviews of teaching, item difficulties and the Ebel standard setting.</p> <p>Moving on to the OSCE findings, the performance patterns also inform clinical tutors and placement supervisors on delivering adaptive clinical teaching and providing tailored support for the two distinct subgroups identified as in need of particular remediation. OSCEgroup1 particularly need very substantial support (e.g. learning opportunities, supervision, feedback, etc.) in ACC, CCC, GOSH, Paediatrics and Psychiatry as they generally achieved much lower than the pass marks in most of the related stations, while OSCEgroup2 could benefit from additional support in ACC, CCC, GOSH, Paediatrics to achieve clear passes in these domains.</p> <p>Students in all three OSCE profiles generally obtained lower scores in ACC and Paediatrics. This generally aligned with their AKT performance patterns in terms of the relative weakness in ACC. Regarding the relatively poor clinical performance in Paediatrics, O'Donoghue et al. ([<reflink idref="bib42" id="ref83">42</reflink>]) makes the point that students generally have limited exposure to cases in this speciality compared to adult medicine in undergraduate studies. As with the AKT, these findings can also be used to inform reviews of teaching and station development.</p> <p>It is noteworthy that even the weakest performers (OSCEgroup1) did relatively well in the integrated stations (Fig. 2). That all sub-groups of students did well on these integrated stations might have wider implications for their validity compared to those stations that are specialty specific and therefore inform curriculum development. The testing context where a specific case scenario is focused in a station may sometimes inhibit students from mobilising their wider knowledge and skills to, for example, comprehensively collect and interpret information from patients (Blaskiewicz et al., [<reflink idref="bib7" id="ref84">7</reflink>]).</p> <p>Our study found a clear overall relationship between AKT and OSCE classifications, but also adds to the evidence suggesting that there can be inconsistency between students' performance in these two assessment formats (Nguyen et al., [<reflink idref="bib39" id="ref85">39</reflink>]). In particular, we found some low AKT achievers did well in the OSCE while some high AKT achievers did not do consistently well in the OSCE. As is well known, a student's limited clinical knowledge and skills can, to some extent, be compensated by relatively good communication skills or other generic skills to achieve a good overall score in OSCE (Boscardin, [<reflink idref="bib8" id="ref86">8</reflink>]; Setyonugroho et al., [<reflink idref="bib48" id="ref87">48</reflink>]). Conversely, a student's OSCE performance may be negatively influenced if they have, for example, a deficit in communication skills, despite having good knowledge skills. According to Attrill et al. ([<reflink idref="bib3" id="ref88">3</reflink>]) and Groene et al. ([<reflink idref="bib18" id="ref89">18</reflink>]), students' communication skills can be moderated by their demographics such as gender and native language background. Also, examiners, depending on their experiences, stringency, etc., can also have important effects on students' OSCE performance (Chong et al., [<reflink idref="bib13" id="ref90">13</reflink>]; Homer, [<reflink idref="bib23" id="ref91">23</reflink>]).</p> <hd id="AN0184191454-15">Pass/fail decision-making</hd> <p>Our work shows that some students in some groups in both the AKT and OSCE did not meet the minimum requirements for some specialities. This is especially the case for the mixed groups which include students who have met the required standard to pass overall, and those who have not (i.e. AKTgroup2 and OSCEgroup2). This finding raises the question of our conception of borderline students and the approach to defining and identifying them (Homer et al., [<reflink idref="bib24" id="ref92">24</reflink>]). Sequential models of assessment are more likely to correctly classify such students through seeking more information on weaker/borderline candidates before final pass/fail decisions are made (Cookson et al., [<reflink idref="bib15" id="ref93">15</reflink>]; Muijtjens et al., [<reflink idref="bib36" id="ref94">36</reflink>]; Pell et al., [<reflink idref="bib44" id="ref95">44</reflink>]). To meet the curriculum requirements for fully preparing students to be qualified doctors (GMC, [<reflink idref="bib16" id="ref96">16</reflink>]), Boscardin ([<reflink idref="bib8" id="ref97">8</reflink>]) argues that cut-off scores for each clinical domain also need to be involved in pass/fail decision making. However, this raises issues around reliability of sub-domains (Sinharay, [<reflink idref="bib49" id="ref98">49</reflink>]), and is likely to increase failure rates.</p> <hd id="AN0184191454-16">Study limitations and future research</hd> <p>Our results of the profile classification in this study were based on the data of one Year 4 cohort in one medical school, and therefore may not be generalisable for other cohorts or other school contexts. However, the approach of profiling students' performance based on summative assessment data as demonstrated in our study, and our findings, could provide models for research focusing on other cohorts or students in different year groups in other institutions, and for research aimed at generating more informative feedback from summative assessment data. If future replication studies with the data of different cohorts were to find similar results (i.e. consistent patterns of relative weaknesses and strengths), this would help confirm specific areas of teaching, curriculum, or placements which need to be reviewed and, perhaps, improved. Finally, our analyses did not account for student demographics (e.g. gender, native language, learning disabilities, etc.), which might have some influence on their (in)consistent performance between the AKT and OSCE. These background factors can be considered in future research investigating and comparing students' performance across assessment formats.</p> <hd id="AN0184191454-17">Author contributions</hd> <p>H.D. conceptualised the study and conducted data analysis and visualisation. H.D. and M.H. both contributed to the methodology and writing, and reviewed, edited and revised the manuscript. Both authors approved the final version of the manuscript for submission.</p> <hd id="AN0184191454-18">Funding</hd> <p>The data employed in this study is not openly available due to the reasons of confidentiality and assessment security.</p> <hd id="AN0184191454-19">Data availability</hd> <p>No datasets were generated or analysed during the current study.</p> <hd id="AN0184191454-20">Declarations</hd> <p></p> <hd id="AN0184191454-21">Ethical approval</hd> <p>The University of Leeds gave permission for this anonymized data to be used for research. The Chair of the University of Leeds School of Medicine research ethics committee and the Head of Assessment at the School confirmed to the authors that formal ethics approval for this study was not required as it involved the use of routinely collected student assessment data which were fully anonymized prior to analysis.</p> <hd id="AN0184191454-22">Competing interests</hd> <p>The authors declare no competing interests.</p> <hd id="AN0184191454-23">Publisher's Note</hd> <p>Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.</p> <ref id="AN0184191454-24"> <title> References </title> <blist> <bibl id="bib1" idref="ref17" type="bt">1</bibl> <bibtext> Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control. 1974; 19; 6: 716-723. 10.1109/TAC.1974.1100705</bibtext> </blist> <blist> <bibl id="bib2" idref="ref16" type="bt">2</bibl> <bibtext> Altshuler L, Wilhite JA, Hardowar K, Crowe R, Hanley K, Kalet A, Zabar S, Gillespie C, Ark T. Understanding medical student paths to communication skills expertise using latent profile analysis. Medical Teacher. 2023; 45; 10: 1140-1147. 10.1080/0142159X.2023.2193303</bibtext> </blist> <blist> <bibl id="bib3" idref="ref32" type="bt">3</bibl> <bibtext> Attrill S, Lincoln M, McAllister S. Supervising international students in clinical placements: Perceptions of experiences and factors influencing competency development. BMC Medical Education. 2016; 16; 1: 180. 10.1186/s12909-016-0702-5</bibtext> </blist> <blist> <bibl id="bib4" idref="ref40" type="bt">4</bibl> <bibtext> Auewarakul C, Downing SM, Jaturatamrong U, Praditsuwan R. Sources of validity evidence for an internal medicine student evaluation system: An evaluative study of assessment methods. Medical Education. 2005; 39; 3: 276-283. 10.1111/j.1365-2929.2005.02090.x</bibtext> </blist> <blist> <bibl id="bib5" idref="ref33" type="bt">5</bibl> <bibtext> Balas M, Scheepers RM, Zador Z, Ibrahim GM, Premji L, Witiw CD. Profiling medical specialties and informing aspiring physicians: A data-driven approach. Advances in Health Sciences Education. 2023. 10.1007/s10459-023-10283-2</bibtext> </blist> <blist> <bibl id="bib6" idref="ref4" type="bt">6</bibl> <bibtext> Black, P, Harrison, C, Lee, C, Marshall, B, & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Open University.</bibtext> </blist> <blist> <bibl id="bib7" idref="ref47" type="bt">7</bibl> <bibtext> Blaskiewicz RJ, Park RS, Chibnall JT, Powell JK. The influence of testing context and clinical rotation order on students' OSCE performance. Academic Medicine. 2004; 79; 6: 597-601. 10.1097/00001888-200406000-00020</bibtext> </blist> <blist> <bibl id="bib8" idref="ref22" type="bt">8</bibl> <bibtext> Boscardin CK. Profiling students for remediation using latent class analysis. Advances in Health Sciences Education. 2012; 17; 1: 55-63. 10.1007/s10459-011-9293-4</bibtext> </blist> <blist> <bibl id="bib9" idref="ref26" type="bt">9</bibl> <bibtext> Boscardin CK, Bullock J, O'Sullivan P, Hauer K. Profiles for success: Examining the relationship between student profiles and clerkship performance using latent profile analysis. Teaching and Learning in Medicine. 2022; 34; 2: 145-154. 10.1080/10401334.2021.1937180</bibtext> </blist> <blist> <bibtext> Burford B, Whittle V, Vance GH. The relationship between medical student learning opportunities and preparedness for practice: A questionnaire study. BMC Medical Education. 2014; 14; 1: 223. 10.1186/1472-6920-14-223</bibtext> </blist> <blist> <bibtext> Burridge S, Shanmugalingam T, Nawrozzadeh F, Leedham-Green K, Sharif A. A qualitative analysis of junior doctors' journeys to preparedness in acute care. BMC Medical Education. 2020; 20; 1: 12. 10.1186/s12909-020-1929-8</bibtext> </blist> <blist> <bibtext> Chang A, Chou CL, Teherani A, Hauer KE. Clinical skills-related learning goals of senior medical students after performance feedback. Medical Education. 2011; 45; 9: 878-885. 10.1111/j.1365-2923.2011.04015.x</bibtext> </blist> <blist> <bibtext> Chong, L, Taylor, S, Haywood, M, Adelstein, B. A, & Shulruf, B. (2017). The sights and insights of examiners in objective structured clinical examinations. Journal of Educational Evaluation for Health Professions, 14. https://doi.org/10.3352/jeehp.2017.14.34.</bibtext> </blist> <blist> <bibtext> Cizek, G. J, & Bunch, M. B. (2007). Standard setting. SAGE.</bibtext> </blist> <blist> <bibtext> Cookson J, Crossley J, Fagan G, McKendree J, Mohsen A. A final clinical examination using a sequential design to improve cost-effectiveness. Medical Education. 2011; 45; 7: 741-747. 10.1111/j.1365-2923.2010.03926.x</bibtext> </blist> <blist> <bibtext> General Medical Council (2015). Promoting excellence: standards for medical education and training. Retrieved October 23, 2023, from https://<ulink href="http://www.gmc-uk.org/education/standards-guidance-and-curricula/standards-and-outcomes/promoting-excellence">www.gmc-uk.org/education/standards-guidance-and-curricula/standards-and-outcomes/promoting-excellence</ulink>.</bibtext> </blist> <blist> <bibtext> General Medical Council (2022). Guidance on undergraduate clinical placements. Retrieved June 2, 2024, from https://<ulink href="http://www.gmc-uk.org/education/standards-guidance-and-curricula/guidance/undergraduate-clinical-placements/guidance-on-undergraduate-clinical-palcements">www.gmc-uk.org/education/standards-guidance-and-curricula/guidance/undergraduate-clinical-placements/guidance-on-undergraduate-clinical-palcements</ulink>.</bibtext> </blist> <blist> <bibtext> Groene OR, Ehrhardt M, Bergelt C. Attitude and communication skills of German medical students. BMC Research Notes. 2022; 15; 1: 11. 10.1186/s13104-021-05901-4</bibtext> </blist> <blist> <bibtext> Harrison CJ, Könings KD, Dannefer EF, Schuwirth LWT, Wass V, van der Vleuten CPM. Factors influencing students' receptivity to formative feedback emerging from different assessment cultures. Perspectives on Medical Education. 2016; 5; 5: 276-284. 10.1007/s40037-016-0297-x</bibtext> </blist> <blist> <bibtext> Harrison CJ, Könings KD, Molyneux A, Schuwirth LWT, Wass V, van der Vleuten CPM. Web-based feedback after summative assessment: How do students engage?. Medical Education. 2013; 47; 7: 734-744. 10.1111/medu.12209</bibtext> </blist> <blist> <bibtext> Harrison CJ, Könings KD, Schuwirth L, Wass V, van der Vleuten C. Barriers to the uptake and use of feedback in the context of summative assessment. Advances in Health Sciences Education. 2015; 20; 1: 229-245. 10.1007/s10459-014-9524-6</bibtext> </blist> <blist> <bibtext> Hattie J, Timperley H. The power of feedback. Review of Educational Research. 2007; 77; 1: 81-112. 10.3102/003465430298487</bibtext> </blist> <blist> <bibtext> Homer M. Towards a more nuanced conceptualisation of differential examiner stringency in OSCEs. Advances in Health Sciences Education. 2023. 10.1007/s10459-023-10289-w</bibtext> </blist> <blist> <bibtext> Homer M, Pell G, Fuller R. Problematizing the concept of the 'borderline' group in performance assessments. Medical Teacher. 2017; 39; 5: 469-475. 10.1080/0142159X.2017.1296563</bibtext> </blist> <blist> <bibtext> Johnson G, Reynard K. Assessment of an objective structured clinical examination (OSCE) for undergraduate students in accident and emergency medicine. Emergency Medicine Journal. 1994; 11; 4: 223-226. 10.1136/emj.11.4.223</bibtext> </blist> <blist> <bibtext> Jung T, Wickrama KS. An introduction to latent class growth analysis and growth mixture modeling. Social and Personality Psychology Compass. 2008; 2; 1: 302-317. 10.1111/j.1751-9004.2007.00054.x</bibtext> </blist> <blist> <bibtext> Lineberry, M. (2020). Assessment affecting learning. In R. Yudkowsky, Y. S. Park, & S. M. Downing (Eds.), Assessment in health professions education (2nd ed, pp. 257–271). Routledge.</bibtext> </blist> <blist> <bibtext> Long, J. S, & Freese, J. (2014). Regression models for categorical dependent variables using Stata (3rd ed.). Stata.</bibtext> </blist> <blist> <bibtext> Lubke GH, Muthén B. Investigating population heterogeneity with factor mixture models. Psychological Methods. 2005; 10; 1: 21-39. 10.1037/1082-989X.10.1.21</bibtext> </blist> <blist> <bibtext> Mak-van der Vossen MC, van Mook WNKA, Kors JM, van Wieringen WN, Peerdeman SM, Croiset G, Kusurkar RA. Distinguishing three unprofessional behavior profiles of medical students using latent class analysis. Academic Medicine. 2016; 91; 9: 1276. 10.1097/ACM.0000000000001206</bibtext> </blist> <blist> <bibtext> Marsh HW, Lüdtke O, Trautwein U, Morin AJS. Classical latent profile analysis of academic self-concept dimensions: Synergy of person- and variable-centered approaches to theoretical models of self-concept. Structural Equation Modeling: A Multidisciplinary Journal. 2009; 16; 2: 191-225. 10.1080/10705510902751010</bibtext> </blist> <blist> <bibtext> Ma, Y. (2021). A cross-cultural study of student self-efficacy profiles and the associated predictors and outcomes using a multigroup latent profile analysis. Studies in Educational Evaluation, 71, 101071. https://doi.org/10.1016/j.stueduc.2021.101071</bibtext> </blist> <blist> <bibtext> McKinley DW, Norcini JJ. How to set standards on performance-based examinations: AMEE Guide 85. Medical Teacher. 2014; 36; 2: 97-110. 10.3109/0142159X.2013.853119</bibtext> </blist> <blist> <bibtext> Medeiros, R. A. (2022). LCAENTROPY: Stata module to calculate entropy for models with categorical latent variables. Statistical Software Components S459118, Boston College Department of Economics, revised 13 Jun 2023.</bibtext> </blist> <blist> <bibtext> Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine. 1990; 65; 9 Suppl: 63-67. 10.1097/00001888-199009000-00045</bibtext> </blist> <blist> <bibtext> Muijtjens AMM, van Vollenhoven FHM, van Luijk SJ, van der Vleuten CPM. Sequential testing in the assessment of clinical skills. Academic Medicine. 2000; 75; 4: 369</bibtext> </blist> <blist> <bibtext> Muthén, B. (2004). Latent variable analysis: Growth mixture modeling and related techniques for longitudinal data. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345–368). Sage.</bibtext> </blist> <blist> <bibtext> Muthén BO, Muthén LK. Integrating person-centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcoholism: Clinical & Experimental Research. 2000; 24; 6: 882-891. 10.1111/j.1530-0277.2000.tb02070.x</bibtext> </blist> <blist> <bibtext> Nguyen M, Beltramo G, Audia S, Devilliers H, Laurent A, Blot M. Correlation between objective structured clinical examinations (OSCE) and knowledge examination scores and analysis of factors associated with better OSCE success in fourth and fifth year medicine students at Dijon university hospital. Revue De Medecine Interne. 2023; 44; 6: 274-281. 10.1016/j.revmed.2023.03.009</bibtext> </blist> <blist> <bibtext> Norcini J, Anderson MB, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Mackay P, Roberts MF, Swanson D. 2018 Consensus framework for good assessment. Medical Teacher. 2018; 40; 11: 1102-1109. 10.1080/0142159X.2018.1500016</bibtext> </blist> <blist> <bibtext> Nylund-Gibson K, Choi AY. Ten frequently asked questions about latent class analysis. Translational Issues in Psychological Science. 2018; 4; 4: 440-461. 10.1037/tps0000176</bibtext> </blist> <blist> <bibtext> O'Donoghue D, Davison G, Hanna LJ, McNaughten B, Stevenson M, Thompson A. Calibration of confidence and assessed clinical skills competence in undergraduate paediatric OSCE scenarios: A mixed methods study. BMC Medical Education. 2018; 18; 1: 211. 10.1186/s12909-018-1318-8</bibtext> </blist> <blist> <bibtext> Park YS, Morales A, Ross L, Paniagua M. Reporting subscore profiles using diagnostic classification models in Health professions Education. Evaluation & the Health Professions. 2020; 43; 3: 149-158. 10.1177/0163278719871090</bibtext> </blist> <blist> <bibtext> Pell G, Fuller R, Homer M, Roberts T. Advancing the objective structured clinical examination: Sequential testing in theory and practice. Medical Education. 2013; 47; 6: 569-577. 10.1111/medu.12136</bibtext> </blist> <blist> <bibtext> Petras, H, & Masyn, K. (2010). General growth mixture analysis with antecedents and consequences of change. In A. R. Piquero & d. Weisburd (Eds.), Handbook of quantitative criminology (pp. 69–100). Springer.</bibtext> </blist> <blist> <bibtext> Schuwirth LWT, van der Vleuten CPM. Programmatic assessment: From assessment of learning to assessment for learning. Medical Teacher. 2011; 33; 6: 478-485</bibtext> </blist> <blist> <bibtext> Schwarz G. Estimating the dimension of a model. Annals of Statistics. 1978; 6; 2: 461-464</bibtext> </blist> <blist> <bibtext> Setyonugroho W, Kennedy KM, Kropmans TJB. Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review. Patient Education and Counseling. 2015; 98; 12: 1482-1491. 10.1016/j.pec.2015.06.004</bibtext> </blist> <blist> <bibtext> Sinharay S. How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement. 2010; 47; 2: 150-174</bibtext> </blist> <blist> <bibtext> Spurk D, Hirschi A, Wang M, Valero D, Kauffeld S. Latent profile analysis: A review and how to guide of its application within vocational behavior research. Journal of Vocational Behavior. 2020; 120: 103445. 10.1016/j.jvb.2020.103445</bibtext> </blist> <blist> <bibtext> StataCorp. (2023). Stata Statistical Software: Release 18. StataCorp LLC.</bibtext> </blist> <blist> <bibtext> Tallentire VR, Smith SE, Skinner J, Cameron HS. The preparedness of UK graduates in acute care: A systematic literature review. Postgraduate Medical Journal. 2012; 88; 1041: 365-371. 10.1136/postgradmedj-2011-130232</bibtext> </blist> <blist> <bibtext> VanOrder T, Robbins W, Zemper E. Residency program directors' interview methods and satisfaction with resident selection across multiple specialties. The Journal of the American Osteopathic Association. 2017; 117; 4: 226-232. 10.7556/jaoa.2017.040</bibtext> </blist> <blist> <bibtext> Vermunt, J. K. (2002). Latent class analysis of complex sample survey data. Journal of the American Statistical Association, 97(459), 736–737. https://pure.uvt.nl/ws/portalfiles/portal/487923/jasa.pdf</bibtext> </blist> <blist> <bibtext> Weller BE, Bowen NK, Faubert SJ. Latent class analysis: A guide to best practice. Journal of Black Psychology. 2020; 46; 4: 287-311. 10.1177/0095798420930932</bibtext> </blist> <blist> <bibtext> Wood TJ, Humphrey-Murto SM, Norman GR. Standard setting in a small scale OSCE: A comparison of the modified borderline-group method and the borderline regression method. Advances in Health Sciences Education. 2006; 11; 2: 115-122. 10.1007/s10459-005-7853-1</bibtext> </blist> <blist> <bibtext> Zelesniack E, Oubaid V, Harendza S. Advanced undergraduate medical students' perceptions of basic medical competences and specific competences for different medical specialties– a qualitative study. BMC Medical Education. 2022; 22; 1: 590. 10.1186/s12909-022-03606-1</bibtext> </blist> </ref> <ref id="AN0184191454-25"> <title> Footnotes </title> <blist> <bibtext> Latent profile analysis is based on data of continuous variables, while latent class analysis employs categorical variables.</bibtext> </blist> <blist> <bibtext> The "pass/fail" here and in what follows refers to the classification of students who have passed the part 1 of the sequential AKT/OSCE or not. Students who have not passed part 1 will have to take part 2.</bibtext> </blist> </ref> <aug> <p>By Huiming Ding and Matt Homer</p> <p>Reported by Author; Author</p> </aug> <nolink nlid="nl1" bibid="bib27" firstref="ref1"></nolink> <nolink nlid="nl2" bibid="bib40" firstref="ref2"></nolink> <nolink nlid="nl3" bibid="bib43" firstref="ref3"></nolink> <nolink nlid="nl4" bibid="bib46" firstref="ref6"></nolink> <nolink nlid="nl5" bibid="bib21" firstref="ref7"></nolink> <nolink nlid="nl6" bibid="bib22" firstref="ref8"></nolink> <nolink nlid="nl7" bibid="bib20" firstref="ref9"></nolink> <nolink nlid="nl8" bibid="bib19" firstref="ref11"></nolink> <nolink nlid="nl9" bibid="bib14" firstref="ref12"></nolink> <nolink nlid="nl10" bibid="bib12" firstref="ref14"></nolink> <nolink nlid="nl11" bibid="bib16" firstref="ref15"></nolink> <nolink nlid="nl12" bibid="bib26" firstref="ref18"></nolink> <nolink nlid="nl13" bibid="bib31" firstref="ref19"></nolink> <nolink nlid="nl14" bibid="bib29" firstref="ref20"></nolink> <nolink nlid="nl15" bibid="bib55" firstref="ref21"></nolink> <nolink nlid="nl16" bibid="bib32" firstref="ref23"></nolink> <nolink nlid="nl17" bibid="bib30" firstref="ref29"></nolink> <nolink nlid="nl18" bibid="bib17" firstref="ref35"></nolink> <nolink nlid="nl19" bibid="bib53" firstref="ref36"></nolink> <nolink nlid="nl20" bibid="bib57" firstref="ref37"></nolink> <nolink nlid="nl21" bibid="bib52" firstref="ref38"></nolink> <nolink nlid="nl22" bibid="bib11" firstref="ref39"></nolink> <nolink nlid="nl23" bibid="bib25" firstref="ref41"></nolink> <nolink nlid="nl24" bibid="bib39" firstref="ref42"></nolink> <nolink nlid="nl25" bibid="bib18" firstref="ref44"></nolink> <nolink nlid="nl26" bibid="bib13" firstref="ref45"></nolink> <nolink nlid="nl27" bibid="bib23" firstref="ref46"></nolink> <nolink nlid="nl28" bibid="bib35" firstref="ref48"></nolink> <nolink nlid="nl29" bibid="bib15" firstref="ref49"></nolink> <nolink nlid="nl30" bibid="bib36" firstref="ref50"></nolink> <nolink nlid="nl31" bibid="bib44" firstref="ref51"></nolink> <nolink nlid="nl32" bibid="bib33" firstref="ref53"></nolink> <nolink nlid="nl33" bibid="bib56" firstref="ref54"></nolink> <nolink nlid="nl34" bibid="bib51" firstref="ref55"></nolink> <nolink nlid="nl35" bibid="bib47" firstref="ref57"></nolink> <nolink nlid="nl36" bibid="bib41" firstref="ref58"></nolink> <nolink nlid="nl37" bibid="bib54" firstref="ref59"></nolink> <nolink nlid="nl38" bibid="bib45" firstref="ref62"></nolink> <nolink nlid="nl39" bibid="bib34" firstref="ref63"></nolink> <nolink nlid="nl40" bibid="bib37" firstref="ref65"></nolink> <nolink nlid="nl41" bibid="bib38" firstref="ref67"></nolink> <nolink nlid="nl42" bibid="bib50" firstref="ref69"></nolink> <nolink nlid="nl43" bibid="bib28" firstref="ref70"></nolink> <nolink nlid="nl44" bibid="bib10" firstref="ref82"></nolink> <nolink nlid="nl45" bibid="bib42" firstref="ref83"></nolink> <nolink nlid="nl46" bibid="bib48" firstref="ref87"></nolink> <nolink nlid="nl47" bibid="bib24" firstref="ref92"></nolink> <nolink nlid="nl48" bibid="bib49" firstref="ref98"></nolink>
Header DbId: eric
DbLabel: ERIC
An: EJ1484818
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Tailoring Support Following Summative Assessments: A Latent Profile Analysis of Student Outcomes across Five Medical Specialities
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Huiming+Ding%22">Huiming Ding</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-5747-7796">0000-0002-5747-7796</externalLink>)<br /><searchLink fieldCode="AR" term="%22Matt+Homer%22">Matt Homer</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0002-1161-5938">0000-0002-1161-5938</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Advances+in+Health+Sciences+Education%22"><i>Advances in Health Sciences Education</i></searchLink>. 2025 30(2):459-473.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 15
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Summative+Evaluation%22">Summative Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Profiles%22">Profiles</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Outcomes+of+Education%22">Outcomes of Education</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+Education%22">Medical Education</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+Students%22">Medical Students</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Objective+Tests%22">Objective Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Specialization%22">Specialization</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s10459-024-10357-9
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1382-4996<br />1573-1677
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Summative assessments are often underused for feedback, despite them being rich with data of students' applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students' performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,…). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1484818
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1484818
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10459-024-10357-9
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 459
    Subjects:
      – SubjectFull: Summative Evaluation
        Type: general
      – SubjectFull: Profiles
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Outcomes of Education
        Type: general
      – SubjectFull: Medical Education
        Type: general
      – SubjectFull: Medical Students
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Objective Tests
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Specialization
        Type: general
    Titles:
      – TitleFull: Tailoring Support Following Summative Assessments: A Latent Profile Analysis of Student Outcomes across Five Medical Specialities
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Huiming Ding
      – PersonEntity:
          Name:
            NameFull: Matt Homer
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 04
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1382-4996
            – Type: issn-electronic
              Value: 1573-1677
          Numbering:
            – Type: volume
              Value: 30
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Advances in Health Sciences Education
              Type: main
ResultId 1