Comparing reusable, atomic feedback with classic feedback on a linear equations task using text mining and qualitative techniques.
Saved in:
| Title: | Comparing reusable, atomic feedback with classic feedback on a linear equations task using text mining and qualitative techniques. |
|---|---|
| Authors: | Moons, Filip1,2 (AUTHOR) f.moons@uu.nl, Holvoet, Alexander3 (AUTHOR), Klingbeil, Katrin4 (AUTHOR), Vandervieren, Ellen2 (AUTHOR) |
| Source: | British Journal of Educational Technology. Sep2024, Vol. 55 Issue 5, p2257-2277. 21p. |
| Subject Terms: | *Mathematics teachers, *Pedagogical content knowledge, *Word frequency, Text mining, Linear equations |
| Abstract: | In this crossover experiment, we investigated the impact of a statement bank, enabling the reuse of previously written feedback (SA condition), on 45 math teachers' feedback for 60 completed linear equation tests, compared to traditional pen‐and‐paper feedback (PP condition). In the SA condition, teachers were encouraged to use atomic feedback, a set of formulation requirements that makes feedback items significantly more reusable. A previous study found that significantly more feedback was written in the SA condition but did not investigate the content of the feedback. To address this gap, we employed a novel approach of combining text mining with qualitative methods. Results indicate similar wording and sentiments in both conditions. However, SA feedback was more elaborate yet general, focusing on major and minor strengths and deficits, while PP feedback was shorter but more concrete, emphasising main issues. Despite low feedback quality in both conditions, the statement bank led to less effective diagnostic activities, implying that teachers' careless use of statement banks, although convenient, might lead to lower‐quality feedback. Practitioner notesWhat is already known about this topic High‐quality feedback should strike a balance between the volume and focus on the main issues, as more feedback does not necessarily equate to better feedback. Feedback should analyse a student's solution whenever possible: interpreting mistakes and communicating that interpretation as feedback.Text mining identifies meaningful patterns and new insights in text using computer algorithms.When teachers can reuse already given feedback using a software tool (statement bank), they tend to write more feedback instead of saving time.What this paper adds Feedback is compared when teachers could use a tool to reuse already given feedback (referred to as 'statement banks') versus a scenario without such a tool. Both approaches observed similar word frequencies, sentiments and amounts of erroneous, descriptive and corrective feedback. However, feedback with a statement bank tended to be more elaborate yet less specific to individual student solutions. In contrast, feedback without the tool was shorter but more concrete, focusing on main issues. Overall, the tool for reusing feedback directed teachers towards less effective diagnostic activities.The paper introduces a novel methodological approach by combining text mining with qualitative techniques in educational research. While text mining provides an overall understanding of differences and similarities in feedback approaches, qualitative methods are essential for in‐depth analysis of content characteristics and feedback quality.Implications for practice and/or policy Statement banks can support teachers by giving more feedback, but in order to improve feedback quality, further measures are necessary (eg, improving pedagogical content knowledge).Teachers may not confuse handiness with quality: statement banks can help, but when used carelessly, teachers tend to describe and correct students' work instead of analysing underlying (mis‐)conceptions using it. Continued attention to feedback quality remains necessary when using such tools. [ABSTRACT FROM AUTHOR] |
| Copyright of British Journal of Educational Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 178994761 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Comparing reusable, atomic feedback with classic feedback on a linear equations task using text mining and qualitative techniques. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Moons%2C+Filip%22">Moons, Filip</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> f.moons@uu.nl</i><br /><searchLink fieldCode="AR" term="%22Holvoet%2C+Alexander%22">Holvoet, Alexander</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Klingbeil%2C+Katrin%22">Klingbeil, Katrin</searchLink><relatesTo>4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Vandervieren%2C+Ellen%22">Vandervieren, Ellen</searchLink><relatesTo>2</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22British+Journal+of+Educational+Technology%22">British Journal of Educational Technology</searchLink>. Sep2024, Vol. 55 Issue 5, p2257-2277. 21p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Mathematics+teachers%22">Mathematics teachers</searchLink><br />*<searchLink fieldCode="DE" term="%22Pedagogical+content+knowledge%22">Pedagogical content knowledge</searchLink><br />*<searchLink fieldCode="DE" term="%22Word+frequency%22">Word frequency</searchLink><br /><searchLink fieldCode="DE" term="%22Text+mining%22">Text mining</searchLink><br /><searchLink fieldCode="DE" term="%22Linear+equations%22">Linear equations</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: In this crossover experiment, we investigated the impact of a statement bank, enabling the reuse of previously written feedback (SA condition), on 45 math teachers' feedback for 60 completed linear equation tests, compared to traditional pen‐and‐paper feedback (PP condition). In the SA condition, teachers were encouraged to use atomic feedback, a set of formulation requirements that makes feedback items significantly more reusable. A previous study found that significantly more feedback was written in the SA condition but did not investigate the content of the feedback. To address this gap, we employed a novel approach of combining text mining with qualitative methods. Results indicate similar wording and sentiments in both conditions. However, SA feedback was more elaborate yet general, focusing on major and minor strengths and deficits, while PP feedback was shorter but more concrete, emphasising main issues. Despite low feedback quality in both conditions, the statement bank led to less effective diagnostic activities, implying that teachers' careless use of statement banks, although convenient, might lead to lower‐quality feedback. Practitioner notesWhat is already known about this topic High‐quality feedback should strike a balance between the volume and focus on the main issues, as more feedback does not necessarily equate to better feedback. Feedback should analyse a student's solution whenever possible: interpreting mistakes and communicating that interpretation as feedback.Text mining identifies meaningful patterns and new insights in text using computer algorithms.When teachers can reuse already given feedback using a software tool (statement bank), they tend to write more feedback instead of saving time.What this paper adds Feedback is compared when teachers could use a tool to reuse already given feedback (referred to as 'statement banks') versus a scenario without such a tool. Both approaches observed similar word frequencies, sentiments and amounts of erroneous, descriptive and corrective feedback. However, feedback with a statement bank tended to be more elaborate yet less specific to individual student solutions. In contrast, feedback without the tool was shorter but more concrete, focusing on main issues. Overall, the tool for reusing feedback directed teachers towards less effective diagnostic activities.The paper introduces a novel methodological approach by combining text mining with qualitative techniques in educational research. While text mining provides an overall understanding of differences and similarities in feedback approaches, qualitative methods are essential for in‐depth analysis of content characteristics and feedback quality.Implications for practice and/or policy Statement banks can support teachers by giving more feedback, but in order to improve feedback quality, further measures are necessary (eg, improving pedagogical content knowledge).Teachers may not confuse handiness with quality: statement banks can help, but when used carelessly, teachers tend to describe and correct students' work instead of analysing underlying (mis‐)conceptions using it. Continued attention to feedback quality remains necessary when using such tools. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of British Journal of Educational Technology is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=178994761 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1111/bjet.13447 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 21 StartPage: 2257 Subjects: – SubjectFull: Mathematics teachers Type: general – SubjectFull: Pedagogical content knowledge Type: general – SubjectFull: Word frequency Type: general – SubjectFull: Text mining Type: general – SubjectFull: Linear equations Type: general Titles: – TitleFull: Comparing reusable, atomic feedback with classic feedback on a linear equations task using text mining and qualitative techniques. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Moons, Filip – PersonEntity: Name: NameFull: Holvoet, Alexander – PersonEntity: Name: NameFull: Klingbeil, Katrin – PersonEntity: Name: NameFull: Vandervieren, Ellen IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 09 Text: Sep2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 00071013 Numbering: – Type: volume Value: 55 – Type: issue Value: 5 Titles: – TitleFull: British Journal of Educational Technology Type: main |
| ResultId | 1 |
