Optimal Regularization for a Data Source.
Saved in:
| Title: | Optimal Regularization for a Data Source. |
|---|---|
| Authors: | Leong, Oscar1 (AUTHOR) oleong@stat.ucla.edu, O' Reilly, Eliza2 (AUTHOR), Soh, Yong Sheng3 (AUTHOR), Chandrasekaran, Venkat4 (AUTHOR) |
| Source: | Foundations of Computational Mathematics. Apr2026, Vol. 26 Issue 2, p889-938. 50p. |
| Subjects: | Mathematical regularization, Inverse problems, Estimation theory, Mathematical optimization, Variational approach (Mathematics), Boltzmann factor |
| Abstract: | In optimization-based approaches to inverse problems and to statistical estimation, it is common to augment criteria that enforce data fidelity with a regularizer that promotes desired structural properties in the solution. The choice of a suitable regularizer is typically driven by a combination of prior domain information and computational considerations. Convex regularizers are attractive computationally but they are limited in the types of structure they can promote. On the other hand, nonconvex regularizers are more flexible in the forms of structure they can promote and they have showcased strong empirical performance in some applications, but they come with the computational challenge of solving the associated optimization problems. In this paper, we seek a systematic understanding of the power and the limitations of convex regularization by investigating the following questions: Given a distribution, what is the optimal regularizer for data drawn from the distribution? What properties of a data source govern whether the optimal regularizer is convex? We address these questions for the class of regularizers specified by functionals that are continuous, positively homogeneous, and positive away from the origin. We say that a regularizer is optimal for a data distribution if the Gibbs density with energy given by the regularizer maximizes the population likelihood (or equivalently, minimizes cross-entropy loss) over all regularizer-induced Gibbs densities. As the regularizers we consider are in one-to-one correspondence with star bodies, we leverage dual Brunn-Minkowski theory to show that a radial function derived from a data distribution is akin to a "computational sufficient statistic" as it is the key quantity for identifying optimal regularizers and for assessing the amenability of a data source to convex regularization. Using tools such as Γ -convergence from variational analysis, we show that our results are robust in the sense that the optimal regularizers for a sample drawn from a distribution converge to their population counterparts as the sample size grows large. Finally, we give generalization guarantees for various families of star bodies that recover previous results for polyhedral regularizers (i.e., dictionary learning) and lead to new ones for a variety of classes of star bodies. [ABSTRACT FROM AUTHOR] |
| Copyright of Foundations of Computational Mathematics is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 193283914 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Optimal Regularization for a Data Source. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Leong%2C+Oscar%22">Leong, Oscar</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> oleong@stat.ucla.edu</i><br /><searchLink fieldCode="AR" term="%22O'+Reilly%2C+Eliza%22">O' Reilly, Eliza</searchLink><relatesTo>2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Soh%2C+Yong+Sheng%22">Soh, Yong Sheng</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Chandrasekaran%2C+Venkat%22">Chandrasekaran, Venkat</searchLink><relatesTo>4</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Foundations+of+Computational+Mathematics%22">Foundations of Computational Mathematics</searchLink>. Apr2026, Vol. 26 Issue 2, p889-938. 50p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Mathematical+regularization%22">Mathematical regularization</searchLink><br /><searchLink fieldCode="DE" term="%22Inverse+problems%22">Inverse problems</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+theory%22">Estimation theory</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+optimization%22">Mathematical optimization</searchLink><br /><searchLink fieldCode="DE" term="%22Variational+approach+%28Mathematics%29%22">Variational approach (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Boltzmann+factor%22">Boltzmann factor</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: In optimization-based approaches to inverse problems and to statistical estimation, it is common to augment criteria that enforce data fidelity with a regularizer that promotes desired structural properties in the solution. The choice of a suitable regularizer is typically driven by a combination of prior domain information and computational considerations. Convex regularizers are attractive computationally but they are limited in the types of structure they can promote. On the other hand, nonconvex regularizers are more flexible in the forms of structure they can promote and they have showcased strong empirical performance in some applications, but they come with the computational challenge of solving the associated optimization problems. In this paper, we seek a systematic understanding of the power and the limitations of convex regularization by investigating the following questions: Given a distribution, what is the optimal regularizer for data drawn from the distribution? What properties of a data source govern whether the optimal regularizer is convex? We address these questions for the class of regularizers specified by functionals that are continuous, positively homogeneous, and positive away from the origin. We say that a regularizer is optimal for a data distribution if the Gibbs density with energy given by the regularizer maximizes the population likelihood (or equivalently, minimizes cross-entropy loss) over all regularizer-induced Gibbs densities. As the regularizers we consider are in one-to-one correspondence with star bodies, we leverage dual Brunn-Minkowski theory to show that a radial function derived from a data distribution is akin to a "computational sufficient statistic" as it is the key quantity for identifying optimal regularizers and for assessing the amenability of a data source to convex regularization. Using tools such as Γ -convergence from variational analysis, we show that our results are robust in the sense that the optimal regularizers for a sample drawn from a distribution converge to their population counterparts as the sample size grows large. Finally, we give generalization guarantees for various families of star bodies that recover previous results for polyhedral regularizers (i.e., dictionary learning) and lead to new ones for a variety of classes of star bodies. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Foundations of Computational Mathematics is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=193283914 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10208-025-09693-y Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 50 StartPage: 889 Subjects: – SubjectFull: Mathematical regularization Type: general – SubjectFull: Inverse problems Type: general – SubjectFull: Estimation theory Type: general – SubjectFull: Mathematical optimization Type: general – SubjectFull: Variational approach (Mathematics) Type: general – SubjectFull: Boltzmann factor Type: general Titles: – TitleFull: Optimal Regularization for a Data Source. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Leong, Oscar – PersonEntity: Name: NameFull: O' Reilly, Eliza – PersonEntity: Name: NameFull: Soh, Yong Sheng – PersonEntity: Name: NameFull: Chandrasekaran, Venkat IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 04 Text: Apr2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 16153375 Numbering: – Type: volume Value: 26 – Type: issue Value: 2 Titles: – TitleFull: Foundations of Computational Mathematics Type: main |
| ResultId | 1 |
