View in EDS

Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.

Saved in:

Bibliographic Details
Title:	Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits.
Authors:	Bauer, Barbara¹ (AUTHOR), Singer, Alexander¹ (AUTHOR), Jakoby, Oliver¹ (AUTHOR), Nickisch, Dirk¹ (AUTHOR), Preuss, Thomas² (AUTHOR), Witt, Johannes² (AUTHOR), Wittwer, Torben¹ (AUTHOR), Gergs, André² (AUTHOR)
Source:	Environmental Toxicology & Chemistry. Jan2025, Vol. 44 Issue 1, p240-250. 11p.
Subjects:	Environmental risk assessment, Time series analysis, Toxicity testing, Plant products, Plant protection
Abstract:	For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment. [ABSTRACT FROM AUTHOR]
	Copyright of Environmental Toxicology & Chemistry is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment. [ABSTRACT FROM AUTHOR]
ISSN:	07307268
DOI:	10.1093/etojnl/vgae015