Highly efficient hierarchical online nonlinear regression using second order methods.
Saved in:
| Title: | Highly efficient hierarchical online nonlinear regression using second order methods. |
|---|---|
| Authors: | Civek, Burak C.1 civek@ee.bilkent.edu.tr, Delibalta, Ibrahim2 ibrahim.delibalta@turktelekom.com.tr, Kozat, Suleyman S.1 kozat@ee.bilkent.edu.tr |
| Source: | Signal Processing. Aug2017, Vol. 137, p22-32. 11p. |
| Subjects: | Nonlinear regression, Algorithms, Decision trees, Linear statistical models, Piecewise linear topology |
| Abstract: | We introduce highly efficient online nonlinear regression algorithms that are suitable for real life applications. We process the data in a truly online manner such that no storage is needed, i.e., the data is discarded after being used. For nonlinear modeling we use a hierarchical piecewise linear approach based on the notion of decision trees where the space of the regressor vectors is adaptively partitioned based on the performance. As the first time in the literature, we learn both the piecewise linear partitioning of the regressor space as well as the linear models in each region using highly effective second order methods, i.e., Newton–Raphson Methods. Hence, we avoid the well known over fitting issues by using piecewise linear models, however, since both the region boundaries as well as the linear models in each region are trained using the second order methods, we achieve substantial performance compared to the state of the art. We demonstrate our gains over the well known benchmark data sets and provide performance results in an individual sequence manner guaranteed to hold without any statistical assumptions. Hence, the introduced algorithms address computational complexity issues widely encountered in real life applications while providing superior guaranteed performance in a strong deterministic sense. [ABSTRACT FROM AUTHOR] |
| Copyright of Signal Processing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 121937059 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Highly efficient hierarchical online nonlinear regression using second order methods. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Civek%2C+Burak+C%2E%22">Civek, Burak C.</searchLink><relatesTo>1</relatesTo><i> civek@ee.bilkent.edu.tr</i><br /><searchLink fieldCode="AR" term="%22Delibalta%2C+Ibrahim%22">Delibalta, Ibrahim</searchLink><relatesTo>2</relatesTo><i> ibrahim.delibalta@turktelekom.com.tr</i><br /><searchLink fieldCode="AR" term="%22Kozat%2C+Suleyman+S%2E%22">Kozat, Suleyman S.</searchLink><relatesTo>1</relatesTo><i> kozat@ee.bilkent.edu.tr</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Signal+Processing%22">Signal Processing</searchLink>. Aug2017, Vol. 137, p22-32. 11p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Nonlinear+regression%22">Nonlinear regression</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Decision+trees%22">Decision trees</searchLink><br /><searchLink fieldCode="DE" term="%22Linear+statistical+models%22">Linear statistical models</searchLink><br /><searchLink fieldCode="DE" term="%22Piecewise+linear+topology%22">Piecewise linear topology</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: We introduce highly efficient online nonlinear regression algorithms that are suitable for real life applications. We process the data in a truly online manner such that no storage is needed, i.e., the data is discarded after being used. For nonlinear modeling we use a hierarchical piecewise linear approach based on the notion of decision trees where the space of the regressor vectors is adaptively partitioned based on the performance. As the first time in the literature, we learn both the piecewise linear partitioning of the regressor space as well as the linear models in each region using highly effective second order methods, i.e., Newton–Raphson Methods. Hence, we avoid the well known over fitting issues by using piecewise linear models, however, since both the region boundaries as well as the linear models in each region are trained using the second order methods, we achieve substantial performance compared to the state of the art. We demonstrate our gains over the well known benchmark data sets and provide performance results in an individual sequence manner guaranteed to hold without any statistical assumptions. Hence, the introduced algorithms address computational complexity issues widely encountered in real life applications while providing superior guaranteed performance in a strong deterministic sense. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Signal Processing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=121937059 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.sigpro.2017.01.029 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 11 StartPage: 22 Subjects: – SubjectFull: Nonlinear regression Type: general – SubjectFull: Algorithms Type: general – SubjectFull: Decision trees Type: general – SubjectFull: Linear statistical models Type: general – SubjectFull: Piecewise linear topology Type: general Titles: – TitleFull: Highly efficient hierarchical online nonlinear regression using second order methods. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Civek, Burak C. – PersonEntity: Name: NameFull: Delibalta, Ibrahim – PersonEntity: Name: NameFull: Kozat, Suleyman S. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 08 Text: Aug2017 Type: published Y: 2017 Identifiers: – Type: issn-print Value: 01651684 Numbering: – Type: volume Value: 137 Titles: – TitleFull: Signal Processing Type: main |
| ResultId | 1 |
