Spectral stochastic gradient method with additional sampling for finite and infinite sums.
Saved in:
| Title: | Spectral stochastic gradient method with additional sampling for finite and infinite sums. |
|---|---|
| Authors: | Krklec Jerinkić, Nataša1 (AUTHOR) natasa.krklec@dmi.uns.ac.rs, Ruggiero, Valeria2 (AUTHOR) rgv@unife.it, Trombini, Ilaria2,3 (AUTHOR) ilaria.trombini@unife.it |
| Source: | Computational Optimization & Applications. Jun2025, Vol. 91 Issue 2, p717-758. 42p. |
| Subjects: | Mathematical forms, Convex functions, Adaptive control systems, Sample size (Statistics), Sampling methods |
| Abstract: | In this paper, we propose a new stochastic gradient method for numerical minimization of finite sums. We also propose a modified version of this method applicable on more general problems referred to as infinite sum problems, where the objective function is in the form of mathematical expectation. The method is based on a strategy to exploit the effectiveness of the well-known Barzilai–Borwein (BB) rules or variants of these (BB-like) rules for updating the step length in the standard gradient method. The proposed method adapts the aforementioned strategy into the stochastic framework by exploiting the same Sample Average Approximations (SAA) estimator of the objective function for several iterations. Furthermore, the sample size is controlled by an additional sampling which also plays a role in accepting the proposed iterate point. Moreover, the number of "inner" iterations with the same sample is also controlled by an adaptive rule which prevents the method from getting stuck with the same estimator for too long. Convergence results are discussed for the finite and infinite sum version, for general and strongly convex objective functions. For the strongly convex case, we provide convergence rate and worst-case complexity analysis. Numerical experiments on well-known datasets for binary classifications show very promising performance of the method, without the need to provide special values for hyperparameters on which the method depends. [ABSTRACT FROM AUTHOR] |
| Copyright of Computational Optimization & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 185240096 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Spectral stochastic gradient method with additional sampling for finite and infinite sums. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Krklec+Jerinkić%2C+Nataša%22">Krklec Jerinkić, Nataša</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> natasa.krklec@dmi.uns.ac.rs</i><br /><searchLink fieldCode="AR" term="%22Ruggiero%2C+Valeria%22">Ruggiero, Valeria</searchLink><relatesTo>2</relatesTo> (AUTHOR)<i> rgv@unife.it</i><br /><searchLink fieldCode="AR" term="%22Trombini%2C+Ilaria%22">Trombini, Ilaria</searchLink><relatesTo>2,3</relatesTo> (AUTHOR)<i> ilaria.trombini@unife.it</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Computational+Optimization+%26+Applications%22">Computational Optimization & Applications</searchLink>. Jun2025, Vol. 91 Issue 2, p717-758. 42p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Mathematical+forms%22">Mathematical forms</searchLink><br /><searchLink fieldCode="DE" term="%22Convex+functions%22">Convex functions</searchLink><br /><searchLink fieldCode="DE" term="%22Adaptive+control+systems%22">Adaptive control systems</searchLink><br /><searchLink fieldCode="DE" term="%22Sample+size+%28Statistics%29%22">Sample size (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Sampling+methods%22">Sampling methods</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: In this paper, we propose a new stochastic gradient method for numerical minimization of finite sums. We also propose a modified version of this method applicable on more general problems referred to as infinite sum problems, where the objective function is in the form of mathematical expectation. The method is based on a strategy to exploit the effectiveness of the well-known Barzilai–Borwein (BB) rules or variants of these (BB-like) rules for updating the step length in the standard gradient method. The proposed method adapts the aforementioned strategy into the stochastic framework by exploiting the same Sample Average Approximations (SAA) estimator of the objective function for several iterations. Furthermore, the sample size is controlled by an additional sampling which also plays a role in accepting the proposed iterate point. Moreover, the number of "inner" iterations with the same sample is also controlled by an adaptive rule which prevents the method from getting stuck with the same estimator for too long. Convergence results are discussed for the finite and infinite sum version, for general and strongly convex objective functions. For the strongly convex case, we provide convergence rate and worst-case complexity analysis. Numerical experiments on well-known datasets for binary classifications show very promising performance of the method, without the need to provide special values for hyperparameters on which the method depends. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Computational Optimization & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=185240096 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10589-025-00664-1 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 42 StartPage: 717 Subjects: – SubjectFull: Mathematical forms Type: general – SubjectFull: Convex functions Type: general – SubjectFull: Adaptive control systems Type: general – SubjectFull: Sample size (Statistics) Type: general – SubjectFull: Sampling methods Type: general Titles: – TitleFull: Spectral stochastic gradient method with additional sampling for finite and infinite sums. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Krklec Jerinkić, Nataša – PersonEntity: Name: NameFull: Ruggiero, Valeria – PersonEntity: Name: NameFull: Trombini, Ilaria IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Text: Jun2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 09266003 Numbering: – Type: volume Value: 91 – Type: issue Value: 2 Titles: – TitleFull: Computational Optimization & Applications Type: main |
| ResultId | 1 |
