View in EDS

De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.

Saved in:
Bibliographic Details
Title: De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.
Authors: Burke, Robin1 (AUTHOR) robin.burke@colorado.edu, Adomavicius, Gediminas2 (AUTHOR) gedas@umn.edu, Bogers, Toine3 (AUTHOR) tobo@itu.dk, Di Noia, Tommaso4 (AUTHOR) tommaso.dinoia@poliba.it, Kowald, Dominik5 (AUTHOR) dkowald@know-center.at, Neidhardt, Julia6 (AUTHOR) julia.neidhardt@tuwien.ac.at, Özgöbek, Özlem7 (AUTHOR) ozlem.ozgobek@ntnu.no, Pera, Maria Soledad8 (AUTHOR) m.s.pera@tudelft.nl, Tintarev, Nava9 (AUTHOR) n.tintarev@maastrichtuniversity.nl, Ziegler, Jürgen10 (AUTHOR) juergen.ziegler@uni-due.de
Source: International Journal of Human-Computer Studies. Sep2025, Vol. 203, pN.PAG-N.PAG. 1p.
Subjects: Recommender systems, Stakeholder theory, Evaluation methodology, Standards, Scholarly method, Values (Ethics), Interpretation (Philosophy), Interdisciplinary research
Abstract: Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved—from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects. • A variety of stakeholders may be impacted by recommender systems. • Holistic evaluation requires considering multiple perspectives. • This area is underexplored in the research literature. • Example metrics and their derivations are described. [ABSTRACT FROM AUTHOR]
Copyright of International Journal of Human-Computer Studies is the property of Academic Press Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved—from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects. • A variety of stakeholders may be impacted by recommender systems. • Holistic evaluation requires considering multiple perspectives. • This area is underexplored in the research literature. • Example metrics and their derivations are described. [ABSTRACT FROM AUTHOR]
ISSN:10715819
DOI:10.1016/j.ijhcs.2025.103560