View in EDS HTML Full Text PDF Full Text

Learning to communicate using a communication critic and counterfactual reasoning.

Saved in:
Bibliographic Details
Title: Learning to communicate using a communication critic and counterfactual reasoning.
Authors: Vanneste, Simon1 (AUTHOR) simon.vanneste@uantwerp.be, Vanneste, Astrid1 (AUTHOR) astrid.vanneste@uantwerpen.be, Mets, Kevin1 (AUTHOR) kevin.mets@uantwerp.be, De Schepper, Tom2 (AUTHOR) tom.deschepper@uantwerp.be, Anwar, Ali1 (AUTHOR) ali.anwar@uantwerp.be, Mercelis, Siegfried1 (AUTHOR) siegfried.mercelis@uantwerp.be, Hellinckx, Peter3 (AUTHOR) peter.hellinckx@uantwerp.be
Source: Neural Computing & Applications. Nov2025, Vol. 37 Issue 31, p25645-25662. 18p.
Subjects: Reinforcement learning, Communication models, Counterfactuals (Logic), Scalability, Information theory, Loss functions (Statistics)
Abstract: Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q-function, is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. As the exact method to create the communication Q-function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment. [ABSTRACT FROM AUTHOR]
Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 188776327
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Learning to communicate using a communication critic and counterfactual reasoning.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Vanneste%2C+Simon%22">Vanneste, Simon</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> simon.vanneste@uantwerp.be</i><br /><searchLink fieldCode="AR" term="%22Vanneste%2C+Astrid%22">Vanneste, Astrid</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> astrid.vanneste@uantwerpen.be</i><br /><searchLink fieldCode="AR" term="%22Mets%2C+Kevin%22">Mets, Kevin</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> kevin.mets@uantwerp.be</i><br /><searchLink fieldCode="AR" term="%22De+Schepper%2C+Tom%22">De Schepper, Tom</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> tom.deschepper@uantwerp.be</i><br /><searchLink fieldCode="AR" term="%22Anwar%2C+Ali%22">Anwar, Ali</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> ali.anwar@uantwerp.be</i><br /><searchLink fieldCode="AR" term="%22Mercelis%2C+Siegfried%22">Mercelis, Siegfried</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> siegfried.mercelis@uantwerp.be</i><br /><searchLink fieldCode="AR" term="%22Hellinckx%2C+Peter%22">Hellinckx, Peter</searchLink><relatesTo>3</relatesTo> (AUTHOR)<i> peter.hellinckx@uantwerp.be</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Neural+Computing+%26+Applications%22">Neural Computing & Applications</searchLink>. Nov2025, Vol. 37 Issue 31, p25645-25662. 18p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Communication+models%22">Communication models</searchLink><br /><searchLink fieldCode="DE" term="%22Counterfactuals+%28Logic%29%22">Counterfactuals (Logic)</searchLink><br /><searchLink fieldCode="DE" term="%22Scalability%22">Scalability</searchLink><br /><searchLink fieldCode="DE" term="%22Information+theory%22">Information theory</searchLink><br /><searchLink fieldCode="DE" term="%22Loss+functions+%28Statistics%29%22">Loss functions (Statistics)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q-function, is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. As the exact method to create the communication Q-function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=188776327
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s00521-024-10598-0
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 18
        StartPage: 25645
    Subjects:
      – SubjectFull: Reinforcement learning
        Type: general
      – SubjectFull: Communication models
        Type: general
      – SubjectFull: Counterfactuals (Logic)
        Type: general
      – SubjectFull: Scalability
        Type: general
      – SubjectFull: Information theory
        Type: general
      – SubjectFull: Loss functions (Statistics)
        Type: general
    Titles:
      – TitleFull: Learning to communicate using a communication critic and counterfactual reasoning.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Vanneste, Simon
      – PersonEntity:
          Name:
            NameFull: Vanneste, Astrid
      – PersonEntity:
          Name:
            NameFull: Mets, Kevin
      – PersonEntity:
          Name:
            NameFull: De Schepper, Tom
      – PersonEntity:
          Name:
            NameFull: Anwar, Ali
      – PersonEntity:
          Name:
            NameFull: Mercelis, Siegfried
      – PersonEntity:
          Name:
            NameFull: Hellinckx, Peter
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Text: Nov2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 09410643
          Numbering:
            – Type: volume
              Value: 37
            – Type: issue
              Value: 31
          Titles:
            – TitleFull: Neural Computing & Applications
              Type: main
ResultId 1