View in EDS HTML Full Text PDF Full Text

Enabling micro-assessments of skills in the simulated setting using temporal artificial intelligence-models.

Saved in:
Bibliographic Details
Title: Enabling micro-assessments of skills in the simulated setting using temporal artificial intelligence-models.
Authors: Bang Andersen, Iben1,2 (AUTHOR), Søndergaard Svendsen, Morten Bo3 (AUTHOR), Risgaard, Anne Line1,2 (AUTHOR) a.risgaard@rn.dk, Sander Danstrup, Christian2,4 (AUTHOR), Todsen, Tobias3,5,6 (AUTHOR), Tolsgaard, Martin G.3,6,7 (AUTHOR), Friis, Mikkel Lønborg1 (AUTHOR)
Source: Medical Teacher. Mar2026, Vol. 48 Issue 3, p415-424. 10p.
Subject Terms: *Predictive tests, *Medical education, *Academic medical centers, *Philosophy of education, *Artificial intelligence, *Teaching methods, *Entry level employees, *Educational technology, *Medical students, *Simulation methods in education, *Research methodology, *National competency-based educational tests, *Automation, *Comparative studies, *Machine learning, Medical personnel, T-test (Statistics), Medical specialties & specialists, Receiver operating characteristic curves, Probability theory, Ultrasonic imaging, Mentoring, Descriptive statistics, Diagnostic errors, Thyroid gland, Software architecture, Data analysis software, Professional competence, Video recording, Sensitivity & specificity (Statistics)
Geographic Terms: Denmark
Abstract: Background: Assessing skills in simulated settings is resource-intensive and lacks validated metrics. Advances in AI offer the potential for automated competence assessment, addressing these limitations. This study aimed to develop and validate a machine learning AI model for automated evaluation during simulation-based thyroid ultrasound (US) training. Methods: Videos from eight experts and 21 novices performing thyroid US on a simulator were analyzed. Frames were processed into sequences of 1, 10, and 50 seconds. A convolutional neural network with a pre-trained ResNet-50 base and a long short-term memory layer analyzed these sequences. The model was trained to distinguish competence levels (competent=1, not competent=0) using fourfold cross-validation, with performance metrics including precision, recall, F1 score, and accuracy. Bayesian updating and adaptive thresholding assessed performance over time. Results: The AI model effectively differentiated expert and novice US performance. The 50-second sequences achieved the highest accuracy (70%) and F1 score (0.76). Experts showed significantly longer durations above the threshold (15.71s) compared to novices (9.31s, p=.030). Conclusions: A long short-term memory-based AI model provides near real-time, automated assessments of competence in US training. Utilizing temporal video data enables detailed micro-assessments of complex procedures, which may enhance interpretability and be applied across various procedural domains. [ABSTRACT FROM AUTHOR]
Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: ehh
DbLabel: Education Research Complete
An: 191766302
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Enabling micro-assessments of skills in the simulated setting using temporal artificial intelligence-models.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Bang+Andersen%2C+Iben%22">Bang Andersen, Iben</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Søndergaard+Svendsen%2C+Morten+Bo%22">Søndergaard Svendsen, Morten Bo</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Risgaard%2C+Anne+Line%22">Risgaard, Anne Line</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> a.risgaard@rn.dk</i><br /><searchLink fieldCode="AR" term="%22Sander+Danstrup%2C+Christian%22">Sander Danstrup, Christian</searchLink><relatesTo>2,4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Todsen%2C+Tobias%22">Todsen, Tobias</searchLink><relatesTo>3,5,6</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Tolsgaard%2C+Martin+G%2E%22">Tolsgaard, Martin G.</searchLink><relatesTo>3,6,7</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Friis%2C+Mikkel+Lønborg%22">Friis, Mikkel Lønborg</searchLink><relatesTo>1</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Medical+Teacher%22">Medical Teacher</searchLink>. Mar2026, Vol. 48 Issue 3, p415-424. 10p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: *<searchLink fieldCode="DE" term="%22Predictive+tests%22">Predictive tests</searchLink><br />*<searchLink fieldCode="DE" term="%22Medical+education%22">Medical education</searchLink><br />*<searchLink fieldCode="DE" term="%22Academic+medical+centers%22">Academic medical centers</searchLink><br />*<searchLink fieldCode="DE" term="%22Philosophy+of+education%22">Philosophy of education</searchLink><br />*<searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Teaching+methods%22">Teaching methods</searchLink><br />*<searchLink fieldCode="DE" term="%22Entry+level+employees%22">Entry level employees</searchLink><br />*<searchLink fieldCode="DE" term="%22Educational+technology%22">Educational technology</searchLink><br />*<searchLink fieldCode="DE" term="%22Medical+students%22">Medical students</searchLink><br />*<searchLink fieldCode="DE" term="%22Simulation+methods+in+education%22">Simulation methods in education</searchLink><br />*<searchLink fieldCode="DE" term="%22Research+methodology%22">Research methodology</searchLink><br />*<searchLink fieldCode="DE" term="%22National+competency-based+educational+tests%22">National competency-based educational tests</searchLink><br />*<searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br />*<searchLink fieldCode="DE" term="%22Comparative+studies%22">Comparative studies</searchLink><br />*<searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+personnel%22">Medical personnel</searchLink><br /><searchLink fieldCode="DE" term="%22T-test+%28Statistics%29%22">T-test (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+specialties+%26+specialists%22">Medical specialties & specialists</searchLink><br /><searchLink fieldCode="DE" term="%22Receiver+operating+characteristic+curves%22">Receiver operating characteristic curves</searchLink><br /><searchLink fieldCode="DE" term="%22Probability+theory%22">Probability theory</searchLink><br /><searchLink fieldCode="DE" term="%22Ultrasonic+imaging%22">Ultrasonic imaging</searchLink><br /><searchLink fieldCode="DE" term="%22Mentoring%22">Mentoring</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+statistics%22">Descriptive statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnostic+errors%22">Diagnostic errors</searchLink><br /><searchLink fieldCode="DE" term="%22Thyroid+gland%22">Thyroid gland</searchLink><br /><searchLink fieldCode="DE" term="%22Software+architecture%22">Software architecture</searchLink><br /><searchLink fieldCode="DE" term="%22Data+analysis+software%22">Data analysis software</searchLink><br /><searchLink fieldCode="DE" term="%22Professional+competence%22">Professional competence</searchLink><br /><searchLink fieldCode="DE" term="%22Video+recording%22">Video recording</searchLink><br /><searchLink fieldCode="DE" term="%22Sensitivity+%26+specificity+%28Statistics%29%22">Sensitivity & specificity (Statistics)</searchLink>
– Name: SubjectGeographic
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Denmark%22">Denmark</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Assessing skills in simulated settings is resource-intensive and lacks validated metrics. Advances in AI offer the potential for automated competence assessment, addressing these limitations. This study aimed to develop and validate a machine learning AI model for automated evaluation during simulation-based thyroid ultrasound (US) training. Methods: Videos from eight experts and 21 novices performing thyroid US on a simulator were analyzed. Frames were processed into sequences of 1, 10, and 50 seconds. A convolutional neural network with a pre-trained ResNet-50 base and a long short-term memory layer analyzed these sequences. The model was trained to distinguish competence levels (competent=1, not competent=0) using fourfold cross-validation, with performance metrics including precision, recall, F1 score, and accuracy. Bayesian updating and adaptive thresholding assessed performance over time. Results: The AI model effectively differentiated expert and novice US performance. The 50-second sequences achieved the highest accuracy (70%) and F1 score (0.76). Experts showed significantly longer durations above the threshold (15.71s) compared to novices (9.31s, p=.030). Conclusions: A long short-term memory-based AI model provides near real-time, automated assessments of competence in US training. Utilizing temporal video data enables detailed micro-assessments of complex procedures, which may enhance interpretability and be applied across various procedural domains. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=191766302
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1080/0142159X.2025.2555353
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
        StartPage: 415
    Subjects:
      – SubjectFull: Predictive tests
        Type: general
      – SubjectFull: Medical education
        Type: general
      – SubjectFull: Academic medical centers
        Type: general
      – SubjectFull: Philosophy of education
        Type: general
      – SubjectFull: Artificial intelligence
        Type: general
      – SubjectFull: Teaching methods
        Type: general
      – SubjectFull: Entry level employees
        Type: general
      – SubjectFull: Educational technology
        Type: general
      – SubjectFull: Medical students
        Type: general
      – SubjectFull: Simulation methods in education
        Type: general
      – SubjectFull: Research methodology
        Type: general
      – SubjectFull: National competency-based educational tests
        Type: general
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Comparative studies
        Type: general
      – SubjectFull: Machine learning
        Type: general
      – SubjectFull: Medical personnel
        Type: general
      – SubjectFull: T-test (Statistics)
        Type: general
      – SubjectFull: Medical specialties & specialists
        Type: general
      – SubjectFull: Receiver operating characteristic curves
        Type: general
      – SubjectFull: Probability theory
        Type: general
      – SubjectFull: Ultrasonic imaging
        Type: general
      – SubjectFull: Mentoring
        Type: general
      – SubjectFull: Descriptive statistics
        Type: general
      – SubjectFull: Diagnostic errors
        Type: general
      – SubjectFull: Thyroid gland
        Type: general
      – SubjectFull: Software architecture
        Type: general
      – SubjectFull: Data analysis software
        Type: general
      – SubjectFull: Professional competence
        Type: general
      – SubjectFull: Video recording
        Type: general
      – SubjectFull: Sensitivity & specificity (Statistics)
        Type: general
      – SubjectFull: Denmark
        Type: general
    Titles:
      – TitleFull: Enabling micro-assessments of skills in the simulated setting using temporal artificial intelligence-models.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Bang Andersen, Iben
      – PersonEntity:
          Name:
            NameFull: Søndergaard Svendsen, Morten Bo
      – PersonEntity:
          Name:
            NameFull: Risgaard, Anne Line
      – PersonEntity:
          Name:
            NameFull: Sander Danstrup, Christian
      – PersonEntity:
          Name:
            NameFull: Todsen, Tobias
      – PersonEntity:
          Name:
            NameFull: Tolsgaard, Martin G.
      – PersonEntity:
          Name:
            NameFull: Friis, Mikkel Lønborg
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Text: Mar2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 0142159X
          Numbering:
            – Type: volume
              Value: 48
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Medical Teacher
              Type: main
ResultId 1