Multi-resolution spectrogram based multi-branch hybrid attention network for music emotion recognition.
Saved in:
| Title: | Multi-resolution spectrogram based multi-branch hybrid attention network for music emotion recognition. |
|---|---|
| Authors: | Su, Yuping1,2 (AUTHOR) ypsu@snnu.edu.cn, Chai, Ruiting1,2 (AUTHOR) chairt@snnu.edu.cn, Yang, Honghong2,3 (AUTHOR) yanghonghong@snnu.edu.cn, Wu, Xiaojun1,2,3 (AUTHOR) xjwu@snnu.edu.cn, Sun, Weitong4 (AUTHOR) swt@snnu.edu.cn |
| Source: | EURASIP Journal on Audio Speech & Music Processing. 3/28/2026, Vol. 2026 Issue 1, p1-16. 16p. |
| Subjects: | Spectrograms, Emotion recognition, Feature extraction |
| Abstract: | Music emotion recognition (MER) is a critical task in the field of music information retrieval. However, most MER research relies solely on single-scale music spectrograms and fails to consider the complementary effects of spectrograms at different scales. Meanwhile, fully extracting emotion-related information from spectrograms remains a major challenge in MER. In this paper, we propose a hybrid attention model based on multi-resolution spectrograms, named MSMHA. The MSMHA model takes multi-scale Mel-spectrograms as inputs, and each input is fed into a well-designed hybrid attention network. The designed attention network successively includes a low-level feature extraction module, a local feature extraction module based on window attention, a channel attention-based long skip connection module, a high-level feature extraction module, and a branch classifier. After being processed by the hybrid attention network, each branch can fully extract emotion-related semantic features from a spectrogram of the specific resolution and output an emotion-classification probability. Finally, a decision-level weighted fusion strategy is applied to the multi-branch outputs to generate the final classification results. The experimental results on the PMEmo dataset demonstrate that our model is both promising and effective, achieving classification accuracies of 90.9%, 86.36%, and 79.87% on the binary-arousal, binary-valence, and four-quadrant dimensions, respectively. Ablation studies further confirm the effectiveness of both the multi-resolution spectrogram inputs and each module of the hybrid attention network. [ABSTRACT FROM AUTHOR] |
| Copyright of EURASIP Journal on Audio Speech & Music Processing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Music emotion recognition (MER) is a critical task in the field of music information retrieval. However, most MER research relies solely on single-scale music spectrograms and fails to consider the complementary effects of spectrograms at different scales. Meanwhile, fully extracting emotion-related information from spectrograms remains a major challenge in MER. In this paper, we propose a hybrid attention model based on multi-resolution spectrograms, named MSMHA. The MSMHA model takes multi-scale Mel-spectrograms as inputs, and each input is fed into a well-designed hybrid attention network. The designed attention network successively includes a low-level feature extraction module, a local feature extraction module based on window attention, a channel attention-based long skip connection module, a high-level feature extraction module, and a branch classifier. After being processed by the hybrid attention network, each branch can fully extract emotion-related semantic features from a spectrogram of the specific resolution and output an emotion-classification probability. Finally, a decision-level weighted fusion strategy is applied to the multi-branch outputs to generate the final classification results. The experimental results on the PMEmo dataset demonstrate that our model is both promising and effective, achieving classification accuracies of 90.9%, 86.36%, and 79.87% on the binary-arousal, binary-valence, and four-quadrant dimensions, respectively. Ablation studies further confirm the effectiveness of both the multi-resolution spectrogram inputs and each module of the hybrid attention network. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 16874714 |
| DOI: | 10.1186/s13636-026-00460-7 |
