Textual Analysis of the Marie Skłodowska-Curie Actions Evaluation Summary Reports. Assessing Strengths and Weaknesses of Funded and Non-Funded Proposals.
Saved in:
| Title: | Textual Analysis of the Marie Skłodowska-Curie Actions Evaluation Summary Reports. Assessing Strengths and Weaknesses of Funded and Non-Funded Proposals. |
|---|---|
| Authors: | Rodella, Ilaria1 ilaria.rodella@unipd.it, Sciandra, Andrea2,3 andrea.sciandra@unipd.it, Tuzzi, Arjuna2 arjuna.tuzzi@unipd.it |
| Source: | Italian Journal of Sociology of Education. Mar2025, Vol. 17 Issue 1, Preceding p247-265. 20p. |
| Subject Terms: | *Artificial intelligence, *Machine learning, *Content analysis, Language models |
| Abstract: | This study analyses Evaluation Summary Reports (ESRs) of Marie Skłodowska-Curie Actions (MSCA) Individual and Postdoctoral Fellowships proposals at the University of Padua (Unipd), spanning Horizon 2020 and Horizon Europe from 2015 to 2022. The aim is to identify recurring strengths and weaknesses in the evaluation process, recognizing the most important and recurrent features of successful proposals. The use of artificial intelligence is also discussed in the paper. Nearly 400 ESRs were analysed by employing keyword extraction and correspondence analysis (CA) to map relationships between words and variables such as project success. While CA did not clearly distinguish between successful and unsuccessful proposals, machine learning was applied. The coordinates from CA were used to predict project outcomes. Comparisons were made with models using only textual features and those employing transformers, specifically, BERT contextualised embeddings. Results showed that using a Large Language Model (LLM) for text representation improved prediction accuracy compared to other methods. However, it highlighted challenges in interpretability and emphasised the need for explicable methods in the absence of words. Overall, the study provides valuable insights for refining support services and training at Unipd, highlighting the effectiveness of LLMs in prediction while acknowledging the interpretive challenges associated with their use. [ABSTRACT FROM AUTHOR] |
| Copyright of Italian Journal of Sociology of Education is the property of Padova University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: ehh DbLabel: Education Research Complete An: 185293096 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Textual Analysis of the Marie Skłodowska-Curie Actions Evaluation Summary Reports. Assessing Strengths and Weaknesses of Funded and Non-Funded Proposals. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Rodella%2C+Ilaria%22">Rodella, Ilaria</searchLink><relatesTo>1</relatesTo><i> ilaria.rodella@unipd.it</i><br /><searchLink fieldCode="AR" term="%22Sciandra%2C+Andrea%22">Sciandra, Andrea</searchLink><relatesTo>2,3</relatesTo><i> andrea.sciandra@unipd.it</i><br /><searchLink fieldCode="AR" term="%22Tuzzi%2C+Arjuna%22">Tuzzi, Arjuna</searchLink><relatesTo>2</relatesTo><i> arjuna.tuzzi@unipd.it</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Italian+Journal+of+Sociology+of+Education%22">Italian Journal of Sociology of Education</searchLink>. Mar2025, Vol. 17 Issue 1, Preceding p247-265. 20p. – Name: Subject Label: Subject Terms Group: Su Data: *<searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br />*<searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br />*<searchLink fieldCode="DE" term="%22Content+analysis%22">Content analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: This study analyses Evaluation Summary Reports (ESRs) of Marie Skłodowska-Curie Actions (MSCA) Individual and Postdoctoral Fellowships proposals at the University of Padua (Unipd), spanning Horizon 2020 and Horizon Europe from 2015 to 2022. The aim is to identify recurring strengths and weaknesses in the evaluation process, recognizing the most important and recurrent features of successful proposals. The use of artificial intelligence is also discussed in the paper. Nearly 400 ESRs were analysed by employing keyword extraction and correspondence analysis (CA) to map relationships between words and variables such as project success. While CA did not clearly distinguish between successful and unsuccessful proposals, machine learning was applied. The coordinates from CA were used to predict project outcomes. Comparisons were made with models using only textual features and those employing transformers, specifically, BERT contextualised embeddings. Results showed that using a Large Language Model (LLM) for text representation improved prediction accuracy compared to other methods. However, it highlighted challenges in interpretability and emphasised the need for explicable methods in the absence of words. Overall, the study provides valuable insights for refining support services and training at Unipd, highlighting the effectiveness of LLMs in prediction while acknowledging the interpretive challenges associated with their use. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Italian Journal of Sociology of Education is the property of Padova University Press and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=ehh&AN=185293096 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.25430/pupj-IJSE-2025-1-12 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 20 StartPage: 247 Subjects: – SubjectFull: Artificial intelligence Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Content analysis Type: general – SubjectFull: Language models Type: general Titles: – TitleFull: Textual Analysis of the Marie Skłodowska-Curie Actions Evaluation Summary Reports. Assessing Strengths and Weaknesses of Funded and Non-Funded Proposals. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Rodella, Ilaria – PersonEntity: Name: NameFull: Sciandra, Andrea – PersonEntity: Name: NameFull: Tuzzi, Arjuna IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 03 Text: Mar2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 20354983 Numbering: – Type: volume Value: 17 – Type: issue Value: 1 Titles: – TitleFull: Italian Journal of Sociology of Education Type: main |
| ResultId | 1 |
