View in EDS

De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.

Saved in:
Bibliographic Details
Title: De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.
Authors: Burke, Robin1 (AUTHOR) robin.burke@colorado.edu, Adomavicius, Gediminas2 (AUTHOR) gedas@umn.edu, Bogers, Toine3 (AUTHOR) tobo@itu.dk, Di Noia, Tommaso4 (AUTHOR) tommaso.dinoia@poliba.it, Kowald, Dominik5 (AUTHOR) dkowald@know-center.at, Neidhardt, Julia6 (AUTHOR) julia.neidhardt@tuwien.ac.at, Özgöbek, Özlem7 (AUTHOR) ozlem.ozgobek@ntnu.no, Pera, Maria Soledad8 (AUTHOR) m.s.pera@tudelft.nl, Tintarev, Nava9 (AUTHOR) n.tintarev@maastrichtuniversity.nl, Ziegler, Jürgen10 (AUTHOR) juergen.ziegler@uni-due.de
Source: International Journal of Human-Computer Studies. Sep2025, Vol. 203, pN.PAG-N.PAG. 1p.
Subjects: Recommender systems, Stakeholder theory, Evaluation methodology, Standards, Scholarly method, Values (Ethics), Interpretation (Philosophy), Interdisciplinary research
Abstract: Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved—from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects. • A variety of stakeholders may be impacted by recommender systems. • Holistic evaluation requires considering multiple perspectives. • This area is underexplored in the research literature. • Example metrics and their derivations are described. [ABSTRACT FROM AUTHOR]
Copyright of International Journal of Human-Computer Studies is the property of Academic Press Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 187461246
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Burke%2C+Robin%22">Burke, Robin</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> robin.burke@colorado.edu</i><br /><searchLink fieldCode="AR" term="%22Adomavicius%2C+Gediminas%22">Adomavicius, Gediminas</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> gedas@umn.edu</i><br /><searchLink fieldCode="AR" term="%22Bogers%2C+Toine%22">Bogers, Toine</searchLink><relatesTo>3</relatesTo> (AUTHOR)<i> tobo@itu.dk</i><br /><searchLink fieldCode="AR" term="%22Di+Noia%2C+Tommaso%22">Di Noia, Tommaso</searchLink><relatesTo>4</relatesTo> (AUTHOR)<i> tommaso.dinoia@poliba.it</i><br /><searchLink fieldCode="AR" term="%22Kowald%2C+Dominik%22">Kowald, Dominik</searchLink><relatesTo>5</relatesTo> (AUTHOR)<i> dkowald@know-center.at</i><br /><searchLink fieldCode="AR" term="%22Neidhardt%2C+Julia%22">Neidhardt, Julia</searchLink><relatesTo>6</relatesTo> (AUTHOR)<i> julia.neidhardt@tuwien.ac.at</i><br /><searchLink fieldCode="AR" term="%22Özgöbek%2C+Özlem%22">Özgöbek, Özlem</searchLink><relatesTo>7</relatesTo> (AUTHOR)<i> ozlem.ozgobek@ntnu.no</i><br /><searchLink fieldCode="AR" term="%22Pera%2C+Maria+Soledad%22">Pera, Maria Soledad</searchLink><relatesTo>8</relatesTo> (AUTHOR)<i> m.s.pera@tudelft.nl</i><br /><searchLink fieldCode="AR" term="%22Tintarev%2C+Nava%22">Tintarev, Nava</searchLink><relatesTo>9</relatesTo> (AUTHOR)<i> n.tintarev@maastrichtuniversity.nl</i><br /><searchLink fieldCode="AR" term="%22Ziegler%2C+Jürgen%22">Ziegler, Jürgen</searchLink><relatesTo>10</relatesTo> (AUTHOR)<i> juergen.ziegler@uni-due.de</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Human-Computer+Studies%22">International Journal of Human-Computer Studies</searchLink>. Sep2025, Vol. 203, pN.PAG-N.PAG. 1p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Recommender+systems%22">Recommender systems</searchLink><br /><searchLink fieldCode="DE" term="%22Stakeholder+theory%22">Stakeholder theory</searchLink><br /><searchLink fieldCode="DE" term="%22Evaluation+methodology%22">Evaluation methodology</searchLink><br /><searchLink fieldCode="DE" term="%22Standards%22">Standards</searchLink><br /><searchLink fieldCode="DE" term="%22Scholarly+method%22">Scholarly method</searchLink><br /><searchLink fieldCode="DE" term="%22Values+%28Ethics%29%22">Values (Ethics)</searchLink><br /><searchLink fieldCode="DE" term="%22Interpretation+%28Philosophy%29%22">Interpretation (Philosophy)</searchLink><br /><searchLink fieldCode="DE" term="%22Interdisciplinary+research%22">Interdisciplinary research</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved—from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects. • A variety of stakeholders may be impacted by recommender systems. • Holistic evaluation requires considering multiple perspectives. • This area is underexplored in the research literature. • Example metrics and their derivations are described. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of International Journal of Human-Computer Studies is the property of Academic Press Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=187461246
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.ijhcs.2025.103560
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 1
        StartPage: N.PAG
    Subjects:
      – SubjectFull: Recommender systems
        Type: general
      – SubjectFull: Stakeholder theory
        Type: general
      – SubjectFull: Evaluation methodology
        Type: general
      – SubjectFull: Standards
        Type: general
      – SubjectFull: Scholarly method
        Type: general
      – SubjectFull: Values (Ethics)
        Type: general
      – SubjectFull: Interpretation (Philosophy)
        Type: general
      – SubjectFull: Interdisciplinary research
        Type: general
    Titles:
      – TitleFull: De-centering the (Traditional) user: Multistakeholder evaluation of recommender systems.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Burke, Robin
      – PersonEntity:
          Name:
            NameFull: Adomavicius, Gediminas
      – PersonEntity:
          Name:
            NameFull: Bogers, Toine
      – PersonEntity:
          Name:
            NameFull: Di Noia, Tommaso
      – PersonEntity:
          Name:
            NameFull: Kowald, Dominik
      – PersonEntity:
          Name:
            NameFull: Neidhardt, Julia
      – PersonEntity:
          Name:
            NameFull: Özgöbek, Özlem
      – PersonEntity:
          Name:
            NameFull: Pera, Maria Soledad
      – PersonEntity:
          Name:
            NameFull: Tintarev, Nava
      – PersonEntity:
          Name:
            NameFull: Ziegler, Jürgen
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Text: Sep2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 10715819
          Numbering:
            – Type: volume
              Value: 203
          Titles:
            – TitleFull: International Journal of Human-Computer Studies
              Type: main
ResultId 1