View in EDS

A Direct Functional Measure of Text Quality: Did the Reader Understand?

Saved in:
Bibliographic Details
Title: A Direct Functional Measure of Text Quality: Did the Reader Understand?
Language: English
Authors: Joachim Grabowski (ORCID 0000-0002-0243-721X), Moti Mathiebe
Source: Written Communication. 2024 41(2):203-229.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 27
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Education Level: Grade 9
High Schools
Junior High Schools
Middle Schools
Secondary Education
Elementary Education
Grade 5
Intermediate Grades
Higher Education
Postsecondary Education
Descriptors: Foreign Countries, Grade 9, Grade 5, College Students, Accidents, Technical Writing, Pictorial Stimuli, Reader Text Relationship, Cognitive Processes, Writing Skills, Secondary School Students, Writing Ability, Descriptive Writing, Writing Evaluation
Geographic Terms: Germany
DOI: 10.1177/07410883231222952
ISSN: 0741-0883
1552-8472
Abstract: Assessing text quality as an indication of underlying skills still remains challenging; irrespective of the approach, many studies struggle with reliability or validity problems. If writing is considered problem-solving, a report must make the reader understand the described situation and call for its mental reconstruction. Therefore, text quality may not only comprise linguistic aspects but also the cognitive-functional power of a text. The presented study aims at exploring the functionality of students' reporting texts in relation to general text-quality measures, using a corpus of accident reports written by German fifth- and ninth-graders (n = 277) prompted by a pictorial stimulus of a bike accident scenario. An online tool was developed in which 277 university students graphically reenacted the situation from one respective text according to the existence, position, and color of the involved elements. Thereafter, the match of the resulting spatial reconstructions with the original situation was assessed by two raters. While most subscales showed sufficiently high interrater reliabilities, the aggregated functionality score ([alpha] = 0.74) had medium-high correlations with other text-quality ratings and was comparably dependent on grade, education level, and linguistic family background. However, the correlational pattern, regression analysis, and factor analysis showed that the functionality score also contributed unique portions of variance to the assessment of writing skill that were not represented by rating measures. Moreover, the direct indication of whether a text allows for the reader's adequate cognitive representation is evident. Altogether, the approach of indicating text functionality through practical understanding offers a sound, though empirically laborious, alternative for text-quality measurement. Results are discussed with regard to the didactical strategy according to which students can improve their writing when they observe whether others can make use of their texts.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1414128
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1414128
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Direct Functional Measure of Text Quality: Did the Reader Understand?
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Joachim+Grabowski%22">Joachim Grabowski</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-0243-721X">0000-0002-0243-721X</externalLink>)<br /><searchLink fieldCode="AR" term="%22Moti+Mathiebe%22">Moti Mathiebe</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Written+Communication%22"><i>Written Communication</i></searchLink>. 2024 41(2):203-229.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 27
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Grade+9%22">Grade 9</searchLink><br /><searchLink fieldCode="EL" term="%22High+Schools%22">High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Elementary+Education%22">Elementary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Grade+5%22">Grade 5</searchLink><br /><searchLink fieldCode="EL" term="%22Intermediate+Grades%22">Intermediate Grades</searchLink><br /><searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+9%22">Grade 9</searchLink><br /><searchLink fieldCode="DE" term="%22Grade+5%22">Grade 5</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Accidents%22">Accidents</searchLink><br /><searchLink fieldCode="DE" term="%22Technical+Writing%22">Technical Writing</searchLink><br /><searchLink fieldCode="DE" term="%22Pictorial+Stimuli%22">Pictorial Stimuli</searchLink><br /><searchLink fieldCode="DE" term="%22Reader+Text+Relationship%22">Reader Text Relationship</searchLink><br /><searchLink fieldCode="DE" term="%22Cognitive+Processes%22">Cognitive Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Skills%22">Writing Skills</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Ability%22">Writing Ability</searchLink><br /><searchLink fieldCode="DE" term="%22Descriptive+Writing%22">Descriptive Writing</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Germany%22">Germany</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/07410883231222952
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0741-0883<br />1552-8472
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Assessing text quality as an indication of underlying skills still remains challenging; irrespective of the approach, many studies struggle with reliability or validity problems. If writing is considered problem-solving, a report must make the reader understand the described situation and call for its mental reconstruction. Therefore, text quality may not only comprise linguistic aspects but also the cognitive-functional power of a text. The presented study aims at exploring the functionality of students' reporting texts in relation to general text-quality measures, using a corpus of accident reports written by German fifth- and ninth-graders (n = 277) prompted by a pictorial stimulus of a bike accident scenario. An online tool was developed in which 277 university students graphically reenacted the situation from one respective text according to the existence, position, and color of the involved elements. Thereafter, the match of the resulting spatial reconstructions with the original situation was assessed by two raters. While most subscales showed sufficiently high interrater reliabilities, the aggregated functionality score ([alpha] = 0.74) had medium-high correlations with other text-quality ratings and was comparably dependent on grade, education level, and linguistic family background. However, the correlational pattern, regression analysis, and factor analysis showed that the functionality score also contributed unique portions of variance to the assessment of writing skill that were not represented by rating measures. Moreover, the direct indication of whether a text allows for the reader's adequate cognitive representation is evident. Altogether, the approach of indicating text functionality through practical understanding offers a sound, though empirically laborious, alternative for text-quality measurement. Results are discussed with regard to the didactical strategy according to which students can improve their writing when they observe whether others can make use of their texts.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1414128
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1414128
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/07410883231222952
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 27
        StartPage: 203
    Subjects:
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Grade 9
        Type: general
      – SubjectFull: Grade 5
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Accidents
        Type: general
      – SubjectFull: Technical Writing
        Type: general
      – SubjectFull: Pictorial Stimuli
        Type: general
      – SubjectFull: Reader Text Relationship
        Type: general
      – SubjectFull: Cognitive Processes
        Type: general
      – SubjectFull: Writing Skills
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Writing Ability
        Type: general
      – SubjectFull: Descriptive Writing
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Germany
        Type: general
    Titles:
      – TitleFull: A Direct Functional Measure of Text Quality: Did the Reader Understand?
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Joachim Grabowski
      – PersonEntity:
          Name:
            NameFull: Moti Mathiebe
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0741-0883
            – Type: issn-electronic
              Value: 1552-8472
          Numbering:
            – Type: volume
              Value: 41
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Written Communication
              Type: main
ResultId 1