Learning to communicate using a communication critic and counterfactual reasoning.
Saved in:
| Title: | Learning to communicate using a communication critic and counterfactual reasoning. |
|---|---|
| Authors: | Vanneste, Simon1 (AUTHOR) simon.vanneste@uantwerp.be, Vanneste, Astrid1 (AUTHOR) astrid.vanneste@uantwerpen.be, Mets, Kevin1 (AUTHOR) kevin.mets@uantwerp.be, De Schepper, Tom2 (AUTHOR) tom.deschepper@uantwerp.be, Anwar, Ali1 (AUTHOR) ali.anwar@uantwerp.be, Mercelis, Siegfried1 (AUTHOR) siegfried.mercelis@uantwerp.be, Hellinckx, Peter3 (AUTHOR) peter.hellinckx@uantwerp.be |
| Source: | Neural Computing & Applications. Nov2025, Vol. 37 Issue 31, p25645-25662. 18p. |
| Subjects: | Reinforcement learning, Communication models, Counterfactuals (Logic), Scalability, Information theory, Loss functions (Statistics) |
| Abstract: | Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q-function, is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. As the exact method to create the communication Q-function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment. [ABSTRACT FROM AUTHOR] |
| Copyright of Neural Computing & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning. The credit assignment problem, the non-stationarity of the communication environment and the problem of encouraging the agents to be influenced by incoming messages are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Next, the non-stationarity of the communication environment, while learning the communication Q-function, is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. As the exact method to create the communication Q-function can be computationally intensive for a large number of agents, two approximation methods are proposed. Additionally, a social loss function is introduced in order to create influenceable agents, which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the particle environment. Finally, we demonstrate the scalability of MACC in a matrix environment. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 09410643 |
| DOI: | 10.1007/s00521-024-10598-0 |
