Speeding up L-BFGS by direct approximation of the inverse Hessian matrix: Speeding up L-BFGS by direct approximation...: A. Sadeghi-Lotfabadi, K. Ghiasi-Shirazi.
Saved in:
| Title: | Speeding up L-BFGS by direct approximation of the inverse Hessian matrix: Speeding up L-BFGS by direct approximation...: A. Sadeghi-Lotfabadi, K. Ghiasi-Shirazi. |
|---|---|
| Authors: | Sadeghi-Lotfabadi, Ashkan1 (AUTHOR) sadeghia@mail.um.ac.ir, Ghiasi-Shirazi, Kamaledin1 (AUTHOR) k.ghiasi@um.ac.ir |
| Source: | Computational Optimization & Applications. May2025, Vol. 91 Issue 1, p283-310. 28p. |
| Subjects: | SIMD (Computer architecture), Computational mathematics, Quasi-Newton methods, Matrix inversion, Linear operators |
| Abstract: | L-BFGS is one of the widely used quasi-Newton methods. Instead of explicitly storing an approximation H of the inverse Hessian, L-BFGS keeps a limited number of vectors that can be used for computing the product of H by the gradient. However, this computation is sequential, each step depending on the outcome of the previous step. To solve this problem, we propose the Direct L-BFGS (DirL-BFGS) method that, seeing H as a linear operator, directly stores a low-rank plus diagonal (LRPD) representation of H. Employing the LRPD representation enables us to leverage the benefits of vector processing, leading to accelerating and parallelizing the calculations in the form of single instruction, multiple data. We evaluate our proposed method on different quadratic optimization problems and several regression and classification tasks with neural networks. Numerical results show that DirL-BFGS is faster overall than L-BFGS. [ABSTRACT FROM AUTHOR] |
| Copyright of Computational Optimization & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 183974669 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Speeding up L-BFGS by direct approximation of the inverse Hessian matrix: Speeding up L-BFGS by direct approximation...: A. Sadeghi-Lotfabadi, K. Ghiasi-Shirazi. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Sadeghi-Lotfabadi%2C+Ashkan%22">Sadeghi-Lotfabadi, Ashkan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> sadeghia@mail.um.ac.ir</i><br /><searchLink fieldCode="AR" term="%22Ghiasi-Shirazi%2C+Kamaledin%22">Ghiasi-Shirazi, Kamaledin</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> k.ghiasi@um.ac.ir</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Computational+Optimization+%26+Applications%22">Computational Optimization & Applications</searchLink>. May2025, Vol. 91 Issue 1, p283-310. 28p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22SIMD+%28Computer+architecture%29%22">SIMD (Computer architecture)</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+mathematics%22">Computational mathematics</searchLink><br /><searchLink fieldCode="DE" term="%22Quasi-Newton+methods%22">Quasi-Newton methods</searchLink><br /><searchLink fieldCode="DE" term="%22Matrix+inversion%22">Matrix inversion</searchLink><br /><searchLink fieldCode="DE" term="%22Linear+operators%22">Linear operators</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: L-BFGS is one of the widely used quasi-Newton methods. Instead of explicitly storing an approximation H of the inverse Hessian, L-BFGS keeps a limited number of vectors that can be used for computing the product of H by the gradient. However, this computation is sequential, each step depending on the outcome of the previous step. To solve this problem, we propose the Direct L-BFGS (DirL-BFGS) method that, seeing H as a linear operator, directly stores a low-rank plus diagonal (LRPD) representation of H. Employing the LRPD representation enables us to leverage the benefits of vector processing, leading to accelerating and parallelizing the calculations in the form of single instruction, multiple data. We evaluate our proposed method on different quadratic optimization problems and several regression and classification tasks with neural networks. Numerical results show that DirL-BFGS is faster overall than L-BFGS. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Computational Optimization & Applications is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=183974669 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s10589-025-00665-0 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 28 StartPage: 283 Subjects: – SubjectFull: SIMD (Computer architecture) Type: general – SubjectFull: Computational mathematics Type: general – SubjectFull: Quasi-Newton methods Type: general – SubjectFull: Matrix inversion Type: general – SubjectFull: Linear operators Type: general Titles: – TitleFull: Speeding up L-BFGS by direct approximation of the inverse Hessian matrix: Speeding up L-BFGS by direct approximation...: A. Sadeghi-Lotfabadi, K. Ghiasi-Shirazi. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Sadeghi-Lotfabadi, Ashkan – PersonEntity: Name: NameFull: Ghiasi-Shirazi, Kamaledin IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 09266003 Numbering: – Type: volume Value: 91 – Type: issue Value: 1 Titles: – TitleFull: Computational Optimization & Applications Type: main |
| ResultId | 1 |
