ModelXGlue: a benchmarking framework for ML tools in MDE.
Saved in:
| Title: | ModelXGlue: a benchmarking framework for ML tools in MDE. |
|---|---|
| Authors: | López, José Antonio Hernández1 (AUTHOR) jose.antonio.hernandez.lopez@liu.se, Cuadrado, Jesús Sánchez1 (AUTHOR) jesusc@um.es, Rubei, Riccardo2 (AUTHOR) riccardo.rubei@univaq.it, Di Ruscio, Davide2 (AUTHOR) davide.diruscio@univaq.it |
| Source: | Software & Systems Modeling. Aug2025, Vol. 24 Issue 4, p1035-1058. 24p. |
| Subjects: | Machine learning, Model-driven software architecture, Automation software, Benchmarking (Management) |
| Abstract: | The integration of machine learning (ML) into model-driven engineering (MDE) holds the potential to enhance the efficiency of modelers and elevate the quality of modeling tools. However, a consensus is yet to be reached on which MDE tasks can derive substantial benefits from ML and how progress in these tasks should be measured. This paper introduces ModelXGlue , a dedicated benchmarking framework to empower researchers when constructing benchmarks for evaluating the application of ML to address MDE tasks. A benchmark is built by referencing datasets and ML models provided by other researchers, and by selecting an evaluation strategy and a set of metrics. ModelXGlue is designed with automation in mind and each component operates in an isolated execution environment (via Docker containers or Python environments), which allows the execution of approaches implemented with diverse technologies like Java, Python, R, etc. We used ModelXGlue to build reference benchmarks for three distinct MDE tasks: model classification, clustering, and feature name recommendation. To build the benchmarks we integrated existing third-party approaches in ModelXGlue. This shows that ModelXGlue is able to accommodate heterogeneous ML models, MDE tasks and different technological requirements. Moreover, we have obtained, for the first time, comparable results for these tasks. Altogether, it emerges that ModelXGlue is a valuable tool for advancing the understanding and evaluation of ML tools within the context of MDE. [ABSTRACT FROM AUTHOR] |
| Copyright of Software & Systems Modeling is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 186986131 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: ModelXGlue: a benchmarking framework for ML tools in MDE. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22López%2C+José+Antonio+Hernández%22">López, José Antonio Hernández</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> jose.antonio.hernandez.lopez@liu.se</i><br /><searchLink fieldCode="AR" term="%22Cuadrado%2C+Jesús+Sánchez%22">Cuadrado, Jesús Sánchez</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> jesusc@um.es</i><br /><searchLink fieldCode="AR" term="%22Rubei%2C+Riccardo%22">Rubei, Riccardo</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> riccardo.rubei@univaq.it</i><br /><searchLink fieldCode="AR" term="%22Di+Ruscio%2C+Davide%22">Di Ruscio, Davide</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> davide.diruscio@univaq.it</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Software+%26+Systems+Modeling%22">Software & Systems Modeling</searchLink>. Aug2025, Vol. 24 Issue 4, p1035-1058. 24p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Model-driven+software+architecture%22">Model-driven software architecture</searchLink><br /><searchLink fieldCode="DE" term="%22Automation+software%22">Automation software</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmarking+%28Management%29%22">Benchmarking (Management)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The integration of machine learning (ML) into model-driven engineering (MDE) holds the potential to enhance the efficiency of modelers and elevate the quality of modeling tools. However, a consensus is yet to be reached on which MDE tasks can derive substantial benefits from ML and how progress in these tasks should be measured. This paper introduces ModelXGlue , a dedicated benchmarking framework to empower researchers when constructing benchmarks for evaluating the application of ML to address MDE tasks. A benchmark is built by referencing datasets and ML models provided by other researchers, and by selecting an evaluation strategy and a set of metrics. ModelXGlue is designed with automation in mind and each component operates in an isolated execution environment (via Docker containers or Python environments), which allows the execution of approaches implemented with diverse technologies like Java, Python, R, etc. We used ModelXGlue to build reference benchmarks for three distinct MDE tasks: model classification, clustering, and feature name recommendation. To build the benchmarks we integrated existing third-party approaches in ModelXGlue. This shows that ModelXGlue is able to accommodate heterogeneous ML models, MDE tasks and different technological requirements. Moreover, we have obtained, for the first time, comparable results for these tasks. Altogether, it emerges that ModelXGlue is a valuable tool for advancing the understanding and evaluation of ML tools within the context of MDE. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Software & Systems Modeling is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=186986131 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10270-024-01183-z Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 24 StartPage: 1035 Subjects: – SubjectFull: Machine learning Type: general – SubjectFull: Model-driven software architecture Type: general – SubjectFull: Automation software Type: general – SubjectFull: Benchmarking (Management) Type: general Titles: – TitleFull: ModelXGlue: a benchmarking framework for ML tools in MDE. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: López, José Antonio Hernández – PersonEntity: Name: NameFull: Cuadrado, Jesús Sánchez – PersonEntity: Name: NameFull: Rubei, Riccardo – PersonEntity: Name: NameFull: Di Ruscio, Davide IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Text: Aug2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 16191366 Numbering: – Type: volume Value: 24 – Type: issue Value: 4 Titles: – TitleFull: Software & Systems Modeling Type: main |
| ResultId | 1 |
