Leveraging broadcast media subtitle transcripts for automatic speech recognition and subtitling.
Saved in:
| Title: | Leveraging broadcast media subtitle transcripts for automatic speech recognition and subtitling. |
|---|---|
| Authors: | Poncelet, Jakob1 (AUTHOR) jakob.poncelet@kuleuven.be, Van hamme, Hugo1 (AUTHOR) hugo.vanhamme@kuleuven.be |
| Source: | EURASIP Journal on Audio Speech & Music Processing. 3/4/2026, Vol. 2026 Issue 1, p1-20. 20p. |
| Subjects: | Automatic speech recognition, Closed captioning, Broadcasting industry, Dutch language, Machine learning, Low-resource languages, Supervised learning |
| Abstract: | The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for low-resource languages and dialects. This paper explores the integration of weakly supervised transcripts from TV subtitles into automatic speech recognition (ASR) systems, aiming to improve both verbatim transcriptions and automatically generated subtitles. To this end, verbatim data and subtitles are regarded as different domains or languages, due to their distinct characteristics. We propose and compare several end-to-end architectures that are designed to jointly model both modalities with separate or shared encoders and decoders. The proposed methods are able to jointly generate a verbatim transcription and a subtitle. Evaluation on Flemish (Belgian Dutch) demonstrates that a model with cascaded encoders and separate decoders allows to represent the differences between the two data types most efficiently while improving on both domains. Despite differences in domain and linguistic variations, combining verbatim transcripts with subtitle data leads to notable ASR improvements. The proposed methods circumvent the need for extensive preprocessing, filtering and cleaning of the subtitled data, which is commonly used to leverage such data for ASR training. Additionally, experiments with a large-scale subtitle dataset show the scalability of the proposed approach. The methods not only improve ASR accuracy but also generate subtitles that closely match standard written text, offering several potential applications. [ABSTRACT FROM AUTHOR] |
| Copyright of EURASIP Journal on Audio Speech & Music Processing is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for low-resource languages and dialects. This paper explores the integration of weakly supervised transcripts from TV subtitles into automatic speech recognition (ASR) systems, aiming to improve both verbatim transcriptions and automatically generated subtitles. To this end, verbatim data and subtitles are regarded as different domains or languages, due to their distinct characteristics. We propose and compare several end-to-end architectures that are designed to jointly model both modalities with separate or shared encoders and decoders. The proposed methods are able to jointly generate a verbatim transcription and a subtitle. Evaluation on Flemish (Belgian Dutch) demonstrates that a model with cascaded encoders and separate decoders allows to represent the differences between the two data types most efficiently while improving on both domains. Despite differences in domain and linguistic variations, combining verbatim transcripts with subtitle data leads to notable ASR improvements. The proposed methods circumvent the need for extensive preprocessing, filtering and cleaning of the subtitled data, which is commonly used to leverage such data for ASR training. Additionally, experiments with a large-scale subtitle dataset show the scalability of the proposed approach. The methods not only improve ASR accuracy but also generate subtitles that closely match standard written text, offering several potential applications. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 16874714 |
| DOI: | 10.1186/s13636-026-00450-9 |
