Interpretable linear dimensionality reduction based on bias-variance analysis.
Saved in:
| Title: | Interpretable linear dimensionality reduction based on bias-variance analysis. |
|---|---|
| Authors: | Bonetti, Paolo1 (AUTHOR) paolo.bonetti@polimi.it, Metelli, Alberto Maria1 (AUTHOR), Restelli, Marcello1 (AUTHOR) |
| Source: | Data Mining & Knowledge Discovery. Jul2024, Vol. 38 Issue 4, p1713-1781. 69p. |
| Subjects: | Continuous groups, Machine learning, Linear statistical models, Design techniques, Algorithms |
| Abstract: | One of the central issues of several machine learning applications on real data is the choice of the input features. Ideally, the designer should select a small number of the relevant, nonredundant features to preserve the complete information contained in the original dataset, with little collinearity among features. This procedure helps mitigate problems like overfitting and the curse of dimensionality, which arise when dealing with high-dimensional problems. On the other hand, it is not desirable to simply discard some features, since they may still contain information that can be exploited to improve results. Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower dimensional space, possibly considering all the original features. However, the projected features resulting from the application of dimensionality reduction techniques are usually difficult to interpret. In this paper, we seek to design a principled dimensionality reduction approach that maintains the interpretability of the resulting features. Specifically, we propose a bias-variance analysis for linear models and we leverage these theoretical results to design an algorithm, Linear Correlated Features Aggregation (LinCFA), which aggregates groups of continuous features with their average if their correlation is "sufficiently large". In this way, all features are considered, the dimensionality is reduced and the interpretability is preserved. Finally, we provide numerical validations of the proposed algorithm both on synthetic datasets to confirm the theoretical results and on real datasets to show some promising applications. [ABSTRACT FROM AUTHOR] |
| Copyright of Data Mining & Knowledge Discovery is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 178776824 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Interpretable linear dimensionality reduction based on bias-variance analysis. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Bonetti%2C+Paolo%22">Bonetti, Paolo</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> paolo.bonetti@polimi.it</i><br /><searchLink fieldCode="AR" term="%22Metelli%2C+Alberto+Maria%22">Metelli, Alberto Maria</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Restelli%2C+Marcello%22">Restelli, Marcello</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Data+Mining+%26+Knowledge+Discovery%22">Data Mining & Knowledge Discovery</searchLink>. Jul2024, Vol. 38 Issue 4, p1713-1781. 69p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Continuous+groups%22">Continuous groups</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Linear+statistical+models%22">Linear statistical models</searchLink><br /><searchLink fieldCode="DE" term="%22Design+techniques%22">Design techniques</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: One of the central issues of several machine learning applications on real data is the choice of the input features. Ideally, the designer should select a small number of the relevant, nonredundant features to preserve the complete information contained in the original dataset, with little collinearity among features. This procedure helps mitigate problems like overfitting and the curse of dimensionality, which arise when dealing with high-dimensional problems. On the other hand, it is not desirable to simply discard some features, since they may still contain information that can be exploited to improve results. Instead, dimensionality reduction techniques are designed to limit the number of features in a dataset by projecting them into a lower dimensional space, possibly considering all the original features. However, the projected features resulting from the application of dimensionality reduction techniques are usually difficult to interpret. In this paper, we seek to design a principled dimensionality reduction approach that maintains the interpretability of the resulting features. Specifically, we propose a bias-variance analysis for linear models and we leverage these theoretical results to design an algorithm, Linear Correlated Features Aggregation (LinCFA), which aggregates groups of continuous features with their average if their correlation is "sufficiently large". In this way, all features are considered, the dimensionality is reduced and the interpretability is preserved. Finally, we provide numerical validations of the proposed algorithm both on synthetic datasets to confirm the theoretical results and on real datasets to show some promising applications. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Data Mining & Knowledge Discovery is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=178776824 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10618-024-01015-0 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 69 StartPage: 1713 Subjects: – SubjectFull: Continuous groups Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Linear statistical models Type: general – SubjectFull: Design techniques Type: general – SubjectFull: Algorithms Type: general Titles: – TitleFull: Interpretable linear dimensionality reduction based on bias-variance analysis. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Bonetti, Paolo – PersonEntity: Name: NameFull: Metelli, Alberto Maria – PersonEntity: Name: NameFull: Restelli, Marcello IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 07 Text: Jul2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 13845810 Numbering: – Type: volume Value: 38 – Type: issue Value: 4 Titles: – TitleFull: Data Mining & Knowledge Discovery Type: main |
| ResultId | 1 |
