Reinforcement learning control with function approximation via multivariate simplex splines.
Saved in:
| Title: | Reinforcement learning control with function approximation via multivariate simplex splines. |
|---|---|
| Authors: | Feng, Yiting1 (AUTHOR), Zhou, Ye1 (AUTHOR) zhouye@usm.my, Ho, Hann Woei1 (AUTHOR), Mat Isa, Nor Ashidi2 (AUTHOR) |
| Source: | International Journal of Adaptive Control & Signal Processing. Oct2025, Vol. 39 Issue 10, p2040-2061. 22p. |
| Subjects: | Reinforcement learning, Splines, Nonlinear systems, Inverted pendulum (Control theory), Optimal control theory, Mathematical analysis |
| Abstract: | Summary: In the field of optimal control for continuous nonlinear systems, function approximation methods are often employed to overcome the curse of dimensionality. Compared to other global function approximators like neural networks, multivariate splines can be easily evaluated and adapted on a local basis with linearity in the parameters. In this work, a multivariate spline based reinforcement learning (RL) strategy is proposed for solving the continuous‐time nonlinear control problem. Based on the classic value iteration method, multivariate splines are integrated into RL algorithms to approximate continuous value functions and policy functions from discrete action and value samples. Hence, the determined splines with updated coefficients can be utilized in continuous control of nonlinear systems. In the simulation experiment, the performance of the spline‐based RL control is evaluated in controlling an under‐actuated inverted pendulum. The proposed method is compared with the value iteration based discrete control strategy and the neural network based continuous control strategy. The simulation results indicate that the proposed method based on multivariate splines has better control performance with less state oscillations, energy consumption and convergence time in comparison with discrete value iteration and neural network based RL, and the adoption of simplex splines improves the function approximation efficiency with less computation time than neural network optimization. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Adaptive Control & Signal Processing is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 188759451 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Reinforcement learning control with function approximation via multivariate simplex splines. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Feng%2C+Yiting%22">Feng, Yiting</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Zhou%2C+Ye%22">Zhou, Ye</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> zhouye@usm.my</i><br /><searchLink fieldCode="AR" term="%22Ho%2C+Hann+Woei%22">Ho, Hann Woei</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Mat+Isa%2C+Nor+Ashidi%22">Mat Isa, Nor Ashidi</searchLink><relatesTo>2</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Adaptive+Control+%26+Signal+Processing%22">International Journal of Adaptive Control & Signal Processing</searchLink>. Oct2025, Vol. 39 Issue 10, p2040-2061. 22p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Splines%22">Splines</searchLink><br /><searchLink fieldCode="DE" term="%22Nonlinear+systems%22">Nonlinear systems</searchLink><br /><searchLink fieldCode="DE" term="%22Inverted+pendulum+%28Control+theory%29%22">Inverted pendulum (Control theory)</searchLink><br /><searchLink fieldCode="DE" term="%22Optimal+control+theory%22">Optimal control theory</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+analysis%22">Mathematical analysis</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Summary: In the field of optimal control for continuous nonlinear systems, function approximation methods are often employed to overcome the curse of dimensionality. Compared to other global function approximators like neural networks, multivariate splines can be easily evaluated and adapted on a local basis with linearity in the parameters. In this work, a multivariate spline based reinforcement learning (RL) strategy is proposed for solving the continuous‐time nonlinear control problem. Based on the classic value iteration method, multivariate splines are integrated into RL algorithms to approximate continuous value functions and policy functions from discrete action and value samples. Hence, the determined splines with updated coefficients can be utilized in continuous control of nonlinear systems. In the simulation experiment, the performance of the spline‐based RL control is evaluated in controlling an under‐actuated inverted pendulum. The proposed method is compared with the value iteration based discrete control strategy and the neural network based continuous control strategy. The simulation results indicate that the proposed method based on multivariate splines has better control performance with less state oscillations, energy consumption and convergence time in comparison with discrete value iteration and neural network based RL, and the adoption of simplex splines improves the function approximation efficiency with less computation time than neural network optimization. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Adaptive Control & Signal Processing is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=188759451 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1002/acs.3579 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 22 StartPage: 2040 Subjects: – SubjectFull: Reinforcement learning Type: general – SubjectFull: Splines Type: general – SubjectFull: Nonlinear systems Type: general – SubjectFull: Inverted pendulum (Control theory) Type: general – SubjectFull: Optimal control theory Type: general – SubjectFull: Mathematical analysis Type: general Titles: – TitleFull: Reinforcement learning control with function approximation via multivariate simplex splines. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Feng, Yiting – PersonEntity: Name: NameFull: Zhou, Ye – PersonEntity: Name: NameFull: Ho, Hann Woei – PersonEntity: Name: NameFull: Mat Isa, Nor Ashidi IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 10 Text: Oct2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 08906327 Numbering: – Type: volume Value: 39 – Type: issue Value: 10 Titles: – TitleFull: International Journal of Adaptive Control & Signal Processing Type: main |
| ResultId | 1 |
