View in EDS

A Transformer Model for Manifesto Classification Using Cross-Context Training: An Ecuadorian Case Study.

Saved in:
Bibliographic Details
Title: A Transformer Model for Manifesto Classification Using Cross-Context Training: An Ecuadorian Case Study.
Authors: Barzallo, Fernanda1 (AUTHOR), Baldeon-Calisto, Maria1,2 (AUTHOR) mbaldeonc@usfq.edu.ec, Pérez, Margorie1 (AUTHOR), Moscoso, Maria Emilia1 (AUTHOR), Navarrete, Danny1 (AUTHOR), Riofrío, Daniel2 (AUTHOR), Medina-Peréz, Pablo3 (AUTHOR), Lai-Yuen, Susana K4 (AUTHOR), Benítez, Diego2 (AUTHOR), Peréz, Noel2 (AUTHOR), Moyano, Ricardo Flores2 (AUTHOR), Fierro, Mateo3 (AUTHOR)
Source: Social Science Computer Review. Jun2025, Vol. 43 Issue 3, p578-603. 26p.
Subject Terms: Natural language processing, Transformer models, Databases, Political manifestoes, Factorial experiment designs
Abstract: Content analysis of political manifestos is necessary to understand the policies and proposed actions of a party. However, manually labeling political texts is time-consuming and labor-intensive. Transformer networks have become essential tools for automating this task. Nevertheless, these models require extensive datasets to achieve good performance. This can be a limitation in manifesto classification, where the availability of publicly labeled datasets can be scarce. To address this challenge, in this work, we developed a Transformer network for the classification of manifestos using a cross-domain training strategy. Using the database of the Comparative Manifesto Project, we implemented a fractional factorial experimental design to determine which Spanish-written manifestos form the best training set for Ecuadorian manifesto labeling. Furthermore, we statistically analyzed which Transformer architecture and preprocessing operations improve the model accuracy. The results indicate that creating a training set with manifestos from Spain and Uruguay, along with implementing stemming and lemmatization preprocessing operations, produces the highest classification accuracy. In addition, we found that the DistilBERT and RoBERTa transformer networks perform statistically similarly and consistently well in manifesto classification. Using the cross-context training strategy, DistilBERT and RoBERTa achieve 60.05% and 57.64% accuracy, respectively, in the classification of the Ecuadorian manifesto. Finally, we investigated the effect of the composition of the training set on performance. The experiments demonstrate that training DistilBERT solely with Ecuadorian manifestos achieves the highest accuracy and F1-score. Furthermore, in the absence of the Ecuadorian dataset, competitive performance is achieved by training the model with datasets from Spain and Uruguay. [ABSTRACT FROM AUTHOR]
Copyright of Social Science Computer Review is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
FullText Text:
  Availability: 0
Header DbId: ehh
DbLabel: Education Research Complete
An: 185157262
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Transformer Model for Manifesto Classification Using Cross-Context Training: An Ecuadorian Case Study.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Barzallo%2C+Fernanda%22">Barzallo, Fernanda</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Baldeon-Calisto%2C+Maria%22">Baldeon-Calisto, Maria</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> mbaldeonc@usfq.edu.ec</i><br /><searchLink fieldCode="AR" term="%22Pérez%2C+Margorie%22">Pérez, Margorie</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Moscoso%2C+Maria+Emilia%22">Moscoso, Maria Emilia</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Navarrete%2C+Danny%22">Navarrete, Danny</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Riofrío%2C+Daniel%22">Riofrío, Daniel</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Medina-Peréz%2C+Pablo%22">Medina-Peréz, Pablo</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Lai-Yuen%2C+Susana+K%22">Lai-Yuen, Susana K</searchLink><relatesTo>4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Benítez%2C+Diego%22">Benítez, Diego</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Peréz%2C+Noel%22">Peréz, Noel</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Moyano%2C+Ricardo+Flores%22">Moyano, Ricardo Flores</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Fierro%2C+Mateo%22">Fierro, Mateo</searchLink><relatesTo>3</relatesTo> (AUTHOR)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Social+Science+Computer+Review%22">Social Science Computer Review</searchLink>. Jun2025, Vol. 43 Issue 3, p578-603. 26p.
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Natural+language+processing%22">Natural language processing</searchLink><br /><searchLink fieldCode="DE" term="%22Transformer+models%22">Transformer models</searchLink><br /><searchLink fieldCode="DE" term="%22Databases%22">Databases</searchLink><br /><searchLink fieldCode="DE" term="%22Political+manifestoes%22">Political manifestoes</searchLink><br /><searchLink fieldCode="DE" term="%22Factorial+experiment+designs%22">Factorial experiment designs</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Content analysis of political manifestos is necessary to understand the policies and proposed actions of a party. However, manually labeling political texts is time-consuming and labor-intensive. Transformer networks have become essential tools for automating this task. Nevertheless, these models require extensive datasets to achieve good performance. This can be a limitation in manifesto classification, where the availability of publicly labeled datasets can be scarce. To address this challenge, in this work, we developed a Transformer network for the classification of manifestos using a cross-domain training strategy. Using the database of the Comparative Manifesto Project, we implemented a fractional factorial experimental design to determine which Spanish-written manifestos form the best training set for Ecuadorian manifesto labeling. Furthermore, we statistically analyzed which Transformer architecture and preprocessing operations improve the model accuracy. The results indicate that creating a training set with manifestos from Spain and Uruguay, along with implementing stemming and lemmatization preprocessing operations, produces the highest classification accuracy. In addition, we found that the DistilBERT and RoBERTa transformer networks perform statistically similarly and consistently well in manifesto classification. Using the cross-context training strategy, DistilBERT and RoBERTa achieve 60.05% and 57.64% accuracy, respectively, in the classification of the Ecuadorian manifesto. Finally, we investigated the effect of the composition of the training set on performance. The experiments demonstrate that training DistilBERT solely with Ecuadorian manifestos achieves the highest accuracy and F1-score. Furthermore, in the absence of the Ecuadorian dataset, competitive performance is achieved by training the model with datasets from Spain and Uruguay. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Social Science Computer Review is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=185157262
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/08944393241266220
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 26
        StartPage: 578
    Subjects:
      – SubjectFull: Natural language processing
        Type: general
      – SubjectFull: Transformer models
        Type: general
      – SubjectFull: Databases
        Type: general
      – SubjectFull: Political manifestoes
        Type: general
      – SubjectFull: Factorial experiment designs
        Type: general
    Titles:
      – TitleFull: A Transformer Model for Manifesto Classification Using Cross-Context Training: An Ecuadorian Case Study.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Barzallo, Fernanda
      – PersonEntity:
          Name:
            NameFull: Baldeon-Calisto, Maria
      – PersonEntity:
          Name:
            NameFull: Pérez, Margorie
      – PersonEntity:
          Name:
            NameFull: Moscoso, Maria Emilia
      – PersonEntity:
          Name:
            NameFull: Navarrete, Danny
      – PersonEntity:
          Name:
            NameFull: Riofrío, Daniel
      – PersonEntity:
          Name:
            NameFull: Medina-Peréz, Pablo
      – PersonEntity:
          Name:
            NameFull: Lai-Yuen, Susana K
      – PersonEntity:
          Name:
            NameFull: Benítez, Diego
      – PersonEntity:
          Name:
            NameFull: Peréz, Noel
      – PersonEntity:
          Name:
            NameFull: Moyano, Ricardo Flores
      – PersonEntity:
          Name:
            NameFull: Fierro, Mateo
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 06
              Text: Jun2025
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 08944393
          Numbering:
            – Type: volume
              Value: 43
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Social Science Computer Review
              Type: main
ResultId 1