View in EDS

Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.

Saved in:
Bibliographic Details
Title: Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.
Authors: Bauer, Barbara1 (AUTHOR), Singer, Alexander1 (AUTHOR), Jakoby, Oliver1 (AUTHOR), Nickisch, Dirk1 (AUTHOR), Preuss, Thomas2 (AUTHOR), Witt, Johannes2 (AUTHOR), Wittwer, Torben1 (AUTHOR), Gergs, André2 (AUTHOR)
Source: Environmental Toxicology & Chemistry. Jan2025, Vol. 44 Issue 1, p240-250. 11p.
Subjects: Environmental risk assessment, Time series analysis, Toxicity testing, Plant products, Plant protection
Abstract: For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment. [ABSTRACT FROM AUTHOR]
Copyright of Environmental Toxicology & Chemistry is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 183714190
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Bauer%2C+Barbara%22">Bauer, Barbara</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Singer%2C+Alexander%22">Singer, Alexander</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Jakoby%2C+Oliver%22">Jakoby, Oliver</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Nickisch%2C+Dirk%22">Nickisch, Dirk</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Preuss%2C+Thomas%22">Preuss, Thomas</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Witt%2C+Johannes%22">Witt, Johannes</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Wittwer%2C+Torben%22">Wittwer, Torben</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Gergs%2C+André%22">Gergs, André</searchLink><relatesTo>2</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Environmental+Toxicology+%26+Chemistry%22">Environmental Toxicology & Chemistry</searchLink>. Jan2025, Vol. 44 Issue 1, p240-250. 11p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Environmental+risk+assessment%22">Environmental risk assessment</searchLink><br /><searchLink fieldCode="DE" term="%22Time+series+analysis%22">Time series analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Toxicity+testing%22">Toxicity testing</searchLink><br /><searchLink fieldCode="DE" term="%22Plant+products%22">Plant products</searchLink><br /><searchLink fieldCode="DE" term="%22Plant+protection%22">Plant protection</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Environmental Toxicology & Chemistry is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=183714190
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1093/etojnl/vgae015
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 240
    Subjects:
      – SubjectFull: Environmental risk assessment
        Type: general
      – SubjectFull: Time series analysis
        Type: general
      – SubjectFull: Toxicity testing
        Type: general
      – SubjectFull: Plant products
        Type: general
      – SubjectFull: Plant protection
        Type: general
    Titles:
      – TitleFull: Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Bauer, Barbara
      – PersonEntity:
          Name:
            NameFull: Singer, Alexander
      – PersonEntity:
          Name:
            NameFull: Jakoby, Oliver
      – PersonEntity:
          Name:
            NameFull: Nickisch, Dirk
      – PersonEntity:
          Name:
            NameFull: Preuss, Thomas
      – PersonEntity:
          Name:
            NameFull: Witt, Johannes
      – PersonEntity:
          Name:
            NameFull: Wittwer, Torben
      – PersonEntity:
          Name:
            NameFull: Gergs, André
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Text: Jan2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 07307268
          Numbering:
            – Type: volume
              Value: 44
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Environmental Toxicology & Chemistry
              Type: main
ResultId 1