View in EDS

How to Evaluate Students' Decisions in a Data Comparison Problem: Correct Decision for the Wrong Reasons?

Saved in:
Bibliographic Details
Title: How to Evaluate Students' Decisions in a Data Comparison Problem: Correct Decision for the Wrong Reasons?
Language: English
Authors: Karel Kok (ORCID 0000-0002-4767-1333), Sophia Chroszczinsky (ORCID 0009-0008-2341-7104), Burkhard Priemer (ORCID 0000-0001-5399-7631)
Source: Physical Review Physics Education Research. 2024 20(1).
Availability: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: https://journals.aps.org/prper/
Peer Reviewed: Y
Page Count: 11
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Secondary Education
Higher Education
Postsecondary Education
Descriptors: Secondary School Students, Undergraduate Students, Comparative Analysis, Evaluation Methods, Test Reliability, Knowledge Level, Decision Making, Science Education, Concept Formation, Measurement Objectives, Logical Thinking
DOI: 10.1103/PhysRevPhysEducRes.20.010129
ISSN: 2469-9896
Abstract: Data comparison problems are used in teaching and science education research that focuses on students' ability to compare datasets and their conceptual understanding of measurement uncertainties. However, the evaluation of students' decisions in these problems can pose a problem: e.g., students making a correct decision for the wrong reasons. Three previous studies, that share the same context and data comparison problem but where participants had increasing conceptual knowledge of measurement uncertainties, are revisited. The comparison shows a troublesome result: increasing conceptual knowledge does not lead to better decision making in the data comparison problem. In this research, we have looked into this apparent discrepancy by comparing and reanalyzing the data from these three studies. We have analyzed students' justifications by coding them based on the compared quantity and the deciding criterion, giving a highly detailed insight into what they do when comparing the datasets. The results show clear differences in the quality of the justifications across the studies and by combining the results with the decisions, we could successfully identify four cases of correct and incorrect decisions for right or wrong reasons. This analysis showed a high prevalence of correct decisions for wrong reasons in two of the studies, resolving the discrepancy in the initial comparison of these studies. The implication of our analysis is that simply asking students to make a decision in data comparison problems is not a suitable probe to gauge their ability to compare datasets or their conceptual understanding of measurement uncertainties and a probe like this should always be complemented by an analysis of the justification.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1432197
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1432197
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: How to Evaluate Students' Decisions in a Data Comparison Problem: Correct Decision for the Wrong Reasons?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Karel+Kok%22">Karel Kok</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-4767-1333">0000-0002-4767-1333</externalLink>)<br /><searchLink fieldCode="AR" term="%22Sophia+Chroszczinsky%22">Sophia Chroszczinsky</searchLink> (ORCID <externalLink term="https://orcid.org/0009-0008-2341-7104">0009-0008-2341-7104</externalLink>)<br /><searchLink fieldCode="AR" term="%22Burkhard+Priemer%22">Burkhard Priemer</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0001-5399-7631">0000-0001-5399-7631</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Physical+Review+Physics+Education+Research%22"><i>Physical Review Physics Education Research</i></searchLink>. 2024 20(1).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: https://journals.aps.org/prper/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+Methods%22">Evaluation Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Reliability%22">Test Reliability</searchLink><br /><searchLink fieldCode="DE" term="%22Knowledge+Level%22">Knowledge Level</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+Making%22">Decision Making</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Education%22">Science Education</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+Formation%22">Concept Formation</searchLink><br /><searchLink fieldCode="DE" term="%22Measurement+Objectives%22">Measurement Objectives</searchLink><br /><searchLink fieldCode="DE" term="%22Logical+Thinking%22">Logical Thinking</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1103/PhysRevPhysEducRes.20.010129
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2469-9896
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Data comparison problems are used in teaching and science education research that focuses on students' ability to compare datasets and their conceptual understanding of measurement uncertainties. However, the evaluation of students' decisions in these problems can pose a problem: e.g., students making a correct decision for the wrong reasons. Three previous studies, that share the same context and data comparison problem but where participants had increasing conceptual knowledge of measurement uncertainties, are revisited. The comparison shows a troublesome result: increasing conceptual knowledge does not lead to better decision making in the data comparison problem. In this research, we have looked into this apparent discrepancy by comparing and reanalyzing the data from these three studies. We have analyzed students' justifications by coding them based on the compared quantity and the deciding criterion, giving a highly detailed insight into what they do when comparing the datasets. The results show clear differences in the quality of the justifications across the studies and by combining the results with the decisions, we could successfully identify four cases of correct and incorrect decisions for right or wrong reasons. This analysis showed a high prevalence of correct decisions for wrong reasons in two of the studies, resolving the discrepancy in the initial comparison of these studies. The implication of our analysis is that simply asking students to make a decision in data comparison problems is not a suitable probe to gauge their ability to compare datasets or their conceptual understanding of measurement uncertainties and a probe like this should always be complemented by an analysis of the justification.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1432197
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1432197
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1103/PhysRevPhysEducRes.20.010129
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
    Subjects:
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Evaluation Methods
        Type: general
      – SubjectFull: Test Reliability
        Type: general
      – SubjectFull: Knowledge Level
        Type: general
      – SubjectFull: Decision Making
        Type: general
      – SubjectFull: Science Education
        Type: general
      – SubjectFull: Concept Formation
        Type: general
      – SubjectFull: Measurement Objectives
        Type: general
      – SubjectFull: Logical Thinking
        Type: general
    Titles:
      – TitleFull: How to Evaluate Students' Decisions in a Data Comparison Problem: Correct Decision for the Wrong Reasons?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Karel Kok
      – PersonEntity:
          Name:
            NameFull: Sophia Chroszczinsky
      – PersonEntity:
          Name:
            NameFull: Burkhard Priemer
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-electronic
              Value: 2469-9896
          Numbering:
            – Type: volume
              Value: 20
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Physical Review Physics Education Research
              Type: main
ResultId 1