View in EDS PDF Full Text

Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.

Saved in:
Bibliographic Details
Title: Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.
Authors: Chao Liu1 liuchaoaero@sina.com
Source: IAENG International Journal of Applied Mathematics. Jan2026, Vol. 56 Issue 1, p347-357. 11p.
Subjects: Reinforcement learning, Inverted pendulum (Control theory), Simulation software, Optimization algorithms, Reward (Psychology), MatLab (Computer software), State-space methods
Abstract: This study establishes a virtual physical model of the cart-pole system in CoppeliaSim and develops a Q-learning-based control algorithm integrated with MATLAB. Through comprehensive simulation, this research systematically investigates the impact of several critical design factors on the controller's learning effectiveness. The analysis reveals that the design of the reward function, particularly the implementation of a state-dependent shaped reward, is the most decisive factor for achieving efficient convergence, proving far superior to simple fixed positive reward strategies. Furthermore, the study presents a detailed analysis of the effects of statespace discretization granularity and the sensitivity of key hyperparameters, including the learning rate and discount factor. Using CoppeliaSim for virtual modeling, this work provides a high-fidelity platform that bridges the gap between numerical simulation and the implementation of physical systems. Through this systematic analysis, critical determinants of controller performance are elucidated, providing deep empirical insights and robust design principles for applying classical reinforcement learning to complex control problems in realistic physical simulations. [ABSTRACT FROM AUTHOR]
Copyright of IAENG International Journal of Applied Mathematics is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 190639093
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chao+Liu%22">Chao Liu</searchLink><relatesTo>1</relatesTo><i> liuchaoaero@sina.com</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22IAENG+International+Journal+of+Applied+Mathematics%22">IAENG International Journal of Applied Mathematics</searchLink>. Jan2026, Vol. 56 Issue 1, p347-357. 11p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Inverted+pendulum+%28Control+theory%29%22">Inverted pendulum (Control theory)</searchLink><br /><searchLink fieldCode="DE" term="%22Simulation+software%22">Simulation software</searchLink><br /><searchLink fieldCode="DE" term="%22Optimization+algorithms%22">Optimization algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Reward+%28Psychology%29%22">Reward (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22MatLab+%28Computer+software%29%22">MatLab (Computer software)</searchLink><br /><searchLink fieldCode="DE" term="%22State-space+methods%22">State-space methods</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study establishes a virtual physical model of the cart-pole system in CoppeliaSim and develops a Q-learning-based control algorithm integrated with MATLAB. Through comprehensive simulation, this research systematically investigates the impact of several critical design factors on the controller's learning effectiveness. The analysis reveals that the design of the reward function, particularly the implementation of a state-dependent shaped reward, is the most decisive factor for achieving efficient convergence, proving far superior to simple fixed positive reward strategies. Furthermore, the study presents a detailed analysis of the effects of statespace discretization granularity and the sensitivity of key hyperparameters, including the learning rate and discount factor. Using CoppeliaSim for virtual modeling, this work provides a high-fidelity platform that bridges the gap between numerical simulation and the implementation of physical systems. Through this systematic analysis, critical determinants of controller performance are elucidated, providing deep empirical insights and robust design principles for applying classical reinforcement learning to complex control problems in realistic physical simulations. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of IAENG International Journal of Applied Mathematics is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=190639093
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 347
    Subjects:
      – SubjectFull: Reinforcement learning
        Type: general
      – SubjectFull: Inverted pendulum (Control theory)
        Type: general
      – SubjectFull: Simulation software
        Type: general
      – SubjectFull: Optimization algorithms
        Type: general
      – SubjectFull: Reward (Psychology)
        Type: general
      – SubjectFull: MatLab (Computer software)
        Type: general
      – SubjectFull: State-space methods
        Type: general
    Titles:
      – TitleFull: Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chao Liu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Text: Jan2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 19929978
          Numbering:
            – Type: volume
              Value: 56
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: IAENG International Journal of Applied Mathematics
              Type: main
ResultId 1