View in EDS

Duration versus Accuracy--What Matters for Computerised Adaptive Testing in Schools?

Saved in:
Bibliographic Details
Title: Duration versus Accuracy--What Matters for Computerised Adaptive Testing in Schools?
Language: English
Authors: Nikola Ebenbeck (ORCID 0000-0002-4167-981X), Morten Bastian, Andreas Mühling, Markus Gebhardt
Source: Journal of Computer Assisted Learning. 2024 40(6):3443-3453.
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 11
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Elementary Secondary Education
Descriptors: Computer Assisted Testing, Elementary Secondary Education, Public Schools, Special Schools, Reading Achievement, Reading Comprehension, Reading Skills, Reading Tests, Test Reliability, Timed Tests, Time Factors (Learning), Accuracy, Error of Measurement
DOI: 10.1111/jcal.13074
ISSN: 0266-4909
1365-2729
Abstract: Background: Computerised adaptive tests (CATs) are tests that provide personalised, efficient and accurate measurement while reducing testing time, depending on the desired level of precision. Schools have different types of assessments that can benefit from a significant reduction in testing time to varying degrees, depending on the area of application, but for which the loss of measurement accuracy has a different impact. The implementation of CAT can take several approaches, each of which can potentially affect the resulting test length and accuracy. Objectives: We compare the methods of estimation-based CAT and binary-search-based CAT to determine the extent to which they are suitable for school assessment in terms of their length and accuracy. Methods: This study uses simulations based on empirical data from a cohort of pupils with and without special needs (n = 400) to examine the effects of probabilistic estimation-based CAT and deterministic binary-search-based CAT on the length and accuracy of an adaptive reading test for pupils with different ability levels. Results and Conclusions: Estimation-based CAT leads to a 40% test reduction with an average accuracy of r = 0.96, while binary-search-based CAT leads to a test reduction of up to 88% with an average accuracy of r = 0.83. Both methods demonstrate the applicability of CAT in educational environments. Practical advantages and disadvantages of both methods for learning environments are discussed, as well as which method is best suited for specific assessment needs.
Abstractor: As Provided
Notes: https://osf.io/5hrgp/?view_only=cf95c4dacb5144ef8141e1994707cb77
Entry Date: 2024
Accession Number: EJ1448399
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1448399
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Duration versus Accuracy--What Matters for Computerised Adaptive Testing in Schools?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Nikola+Ebenbeck%22">Nikola Ebenbeck</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4167-981X">0000-0002-4167-981X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Morten+Bastian%22">Morten Bastian</searchLink><br /><searchLink fieldCode="AR" term="%22Andreas+Mühling%22">Andreas Mühling</searchLink><br /><searchLink fieldCode="AR" term="%22Markus+Gebhardt%22">Markus Gebhardt</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Computer+Assisted+Learning%22"><i>Journal of Computer Assisted Learning</i></searchLink>. 2024 40(6):3443-3453.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22Public+Schools%22">Public Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Special+Schools%22">Special Schools</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Achievement%22">Reading Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Comprehension%22">Reading Comprehension</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Skills%22">Reading Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Tests%22">Reading Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Timed+Tests%22">Timed Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Time+Factors+%28Learning%29%22">Time Factors (Learning)</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jcal.13074
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0266-4909<br />1365-2729
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Computerised adaptive tests (CATs) are tests that provide personalised, efficient and accurate measurement while reducing testing time, depending on the desired level of precision. Schools have different types of assessments that can benefit from a significant reduction in testing time to varying degrees, depending on the area of application, but for which the loss of measurement accuracy has a different impact. The implementation of CAT can take several approaches, each of which can potentially affect the resulting test length and accuracy. Objectives: We compare the methods of estimation-based CAT and binary-search-based CAT to determine the extent to which they are suitable for school assessment in terms of their length and accuracy. Methods: This study uses simulations based on empirical data from a cohort of pupils with and without special needs (n = 400) to examine the effects of probabilistic estimation-based CAT and deterministic binary-search-based CAT on the length and accuracy of an adaptive reading test for pupils with different ability levels. Results and Conclusions: Estimation-based CAT leads to a 40% test reduction with an average accuracy of r = 0.96, while binary-search-based CAT leads to a test reduction of up to 88% with an average accuracy of r = 0.83. Both methods demonstrate the applicability of CAT in educational environments. Practical advantages and disadvantages of both methods for learning environments are discussed, as well as which method is best suited for specific assessment needs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://osf.io/5hrgp/?view_only=cf95c4dacb5144ef8141e1994707cb77
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1448399
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1448399
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jcal.13074
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 3443
    Subjects:
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Elementary Secondary Education
        Type: general
      – SubjectFull: Public Schools
        Type: general
      – SubjectFull: Special Schools
        Type: general
      – SubjectFull: Reading Achievement
        Type: general
      – SubjectFull: Reading Comprehension
        Type: general
      – SubjectFull: Reading Skills
        Type: general
      – SubjectFull: Reading Tests
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Timed Tests
        Type: general
      – SubjectFull: Time Factors (Learning)
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
    Titles:
      – TitleFull: Duration versus Accuracy--What Matters for Computerised Adaptive Testing in Schools?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Nikola Ebenbeck
      – PersonEntity:
          Name:
            NameFull: Morten Bastian
      – PersonEntity:
          Name:
            NameFull: Andreas Mühling
      – PersonEntity:
          Name:
            NameFull: Markus Gebhardt
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0266-4909
            – Type: issn-electronic
              Value: 1365-2729
          Numbering:
            – Type: volume
              Value: 40
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Journal of Computer Assisted Learning
              Type: main
ResultId 1