View in EDS PDF Full Text

Empirical Evaluation of a Differentiated Assessment of Data Structures: The Role of Prerequisite Skills

Saved in:
Bibliographic Details
Title: Empirical Evaluation of a Differentiated Assessment of Data Structures: The Role of Prerequisite Skills
Language: English
Authors: Marjahan Begum, Pontus Haglund, Ari Korhonen, Violetta Lonati, Mattia Monga, Filip Strömbäck, Artturi Tilanterä
Source: Informatics in Education. 2024 23(1):57-99.
Availability: Vilnius University Institute of Mathematics and Informatics, Lithuanian Academy of Sciences. Akademjos str. 4, Vilnius LT 08663 Lithuania. Tel: +37-5-21-09300; Fax: +37-5-27-29209; e-mail: info@mii.vu.lt; Web site: https://infedu.vu.lt/journal/INFEDU
Peer Reviewed: Y
Page Count: 43
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Tests/Questionnaires
Education Level: Higher Education
Postsecondary Education
Descriptors: Foreign Countries, College Students, Prerequisites, Computer Science Education, Data Collection, Measurement Objectives, Test Reliability, Alternative Assessment, Evaluation Methods
Geographic Terms: Sweden, Finland
ISSN: 1648-5831
2335-8971
Abstract: There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating between fragilities in prerequisites (in particular, knowledge and skills related to introductory programming courses) and advanced topics. This paper reports on an empirical evaluation of an instrument focusing on data structures, among those proposed by the ITiCSE working group. The evaluation aimed at understanding what fragile knowledge and skills the instrument is actually able to detect and to what extent it is able to differentiate them. Our results support that the instrument is able to distinguish between some specific fragilities (e.g., value vs. reference semantics), but not all of those claimed in the original report. In addition, our findings highlight the role of relevant skills at a level between prerequisite and advanced skills, such as program comprehension and reasoning about constraints. We also suggest ways to improve the questions in the instrument, both by improving the distractors of the multiple-choice questions, and by slightly changing the content or phrasing of the questions. We argue that these improvements will increase the effectiveness of the instrument in assessing prerequisites as a whole, but also to pinpoint specific fragilities.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1428763
Database: ERIC
FullText Links:
  – Type: pdflink
    Url: https://content.ebscohost.com/cds/retrieve?content=AQICAHj0k_4E0hTGH8RJwT4gCJyBsGNe_WN95AvKlDbXJGqwxwFIqI7xO-fWBBXu7QXBwyX_AAAA4zCB4AYJKoZIhvcNAQcGoIHSMIHPAgEAMIHJBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDFzGwJPDSZ7YLrXocAIBEICBm7eab_sFBvCP0Sb7ub01eQdhj3_DYA_25G7FByfgRaoWXQp5WnB8lTcZuGj2XSnfaA33UMTzw1fwEMouQIicdX0eXfpweAru55XnObVd1sfuhhLG6657rvc6nHmwKmHwilOF9hxtpImRl6EiCpBEeZb-nbuX059f6Z_HHIYnQRcsKlKWBm8SlJAERLmWKgV5ZjKEoPnkP7RLHBA3
Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1428763
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1428763
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Empirical Evaluation of a Differentiated Assessment of Data Structures: The Role of Prerequisite Skills
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Marjahan+Begum%22">Marjahan Begum</searchLink><br /><searchLink fieldCode="AR" term="%22Pontus+Haglund%22">Pontus Haglund</searchLink><br /><searchLink fieldCode="AR" term="%22Ari+Korhonen%22">Ari Korhonen</searchLink><br /><searchLink fieldCode="AR" term="%22Violetta+Lonati%22">Violetta Lonati</searchLink><br /><searchLink fieldCode="AR" term="%22Mattia+Monga%22">Mattia Monga</searchLink><br /><searchLink fieldCode="AR" term="%22Filip+Strömbäck%22">Filip Strömbäck</searchLink><br /><searchLink fieldCode="AR" term="%22Artturi+Tilanterä%22">Artturi Tilanterä</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Informatics+in+Education%22"><i>Informatics in Education</i></searchLink>. 2024 23(1):57-99.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Vilnius University Institute of Mathematics and Informatics, Lithuanian Academy of Sciences. Akademjos str. 4, Vilnius LT 08663 Lithuania. Tel: +37-5-21-09300; Fax: +37-5-27-29209; e-mail: info@mii.vu.lt; Web site: https://infedu.vu.lt/journal/INFEDU
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 43
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research<br />Tests/Questionnaires
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Prerequisites%22">Prerequisites</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+Education%22">Computer Science Education</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Collection%22">Data Collection</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement+Objectives%22">Measurement Objectives</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Alternative+Assessment%22">Alternative Assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Sweden%22">Sweden</searchLink><br /><searchLink fieldCode="DE" term="%22Finland%22">Finland</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1648-5831<br />2335-8971
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating between fragilities in prerequisites (in particular, knowledge and skills related to introductory programming courses) and advanced topics. This paper reports on an empirical evaluation of an instrument focusing on data structures, among those proposed by the ITiCSE working group. The evaluation aimed at understanding what fragile knowledge and skills the instrument is actually able to detect and to what extent it is able to differentiate them. Our results support that the instrument is able to distinguish between some specific fragilities (e.g., value vs. reference semantics), but not all of those claimed in the original report. In addition, our findings highlight the role of relevant skills at a level between prerequisite and advanced skills, such as program comprehension and reasoning about constraints. We also suggest ways to improve the questions in the instrument, both by improving the distractors of the multiple-choice questions, and by slightly changing the content or phrasing of the questions. We argue that these improvements will increase the effectiveness of the instrument in assessing prerequisites as a whole, but also to pinpoint specific fragilities.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1428763
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1428763
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 43
        StartPage: 57
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Prerequisites
        Type: general
      – SubjectFull: Computer Science Education
        Type: general
      – SubjectFull: Data Collection
        Type: general
      – SubjectFull: Measurement Objectives
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Alternative Assessment
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Sweden
        Type: general
      – SubjectFull: Finland
        Type: general
    Titles:
      – TitleFull: Empirical Evaluation of a Differentiated Assessment of Data Structures: The Role of Prerequisite Skills
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Marjahan Begum
      – PersonEntity:
          Name:
            NameFull: Pontus Haglund
      – PersonEntity:
          Name:
            NameFull: Ari Korhonen
      – PersonEntity:
          Name:
            NameFull: Violetta Lonati
      – PersonEntity:
          Name:
            NameFull: Mattia Monga
      – PersonEntity:
          Name:
            NameFull: Filip Strömbäck
      – PersonEntity:
          Name:
            NameFull: Artturi Tilanterä
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 1648-5831
            – Type: issn-electronic
              Value: 2335-8971
          Numbering:
            – Type: volume
              Value: 23
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Informatics in Education
              Type: main
ResultId 1