Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.

Saved in:

Bibliographic Details
Title:	Q-Learning-Based Control of Cart-Pole System in the CoppeliaSim.
Authors:	Chao Liu¹ liuchaoaero@sina.com
Source:	IAENG International Journal of Applied Mathematics. Jan2026, Vol. 56 Issue 1, p347-357. 11p.
Subjects:	Reinforcement learning, Inverted pendulum (Control theory), Simulation software, Optimization algorithms, Reward (Psychology), MatLab (Computer software), State-space methods
Abstract:	This study establishes a virtual physical model of the cart-pole system in CoppeliaSim and develops a Q-learning-based control algorithm integrated with MATLAB. Through comprehensive simulation, this research systematically investigates the impact of several critical design factors on the controller's learning effectiveness. The analysis reveals that the design of the reward function, particularly the implementation of a state-dependent shaped reward, is the most decisive factor for achieving efficient convergence, proving far superior to simple fixed positive reward strategies. Furthermore, the study presents a detailed analysis of the effects of statespace discretization granularity and the sensitivity of key hyperparameters, including the learning rate and discount factor. Using CoppeliaSim for virtual modeling, this work provides a high-fidelity platform that bridges the gap between numerical simulation and the implementation of physical systems. Through this systematic analysis, critical determinants of controller performance are elucidated, providing deep empirical insights and robust design principles for applying classical reinforcement learning to complex control problems in realistic physical simulations. [ABSTRACT FROM AUTHOR]
	Copyright of IAENG International Journal of Applied Mathematics is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	This study establishes a virtual physical model of the cart-pole system in CoppeliaSim and develops a Q-learning-based control algorithm integrated with MATLAB. Through comprehensive simulation, this research systematically investigates the impact of several critical design factors on the controller's learning effectiveness. The analysis reveals that the design of the reward function, particularly the implementation of a state-dependent shaped reward, is the most decisive factor for achieving efficient convergence, proving far superior to simple fixed positive reward strategies. Furthermore, the study presents a detailed analysis of the effects of statespace discretization granularity and the sensitivity of key hyperparameters, including the learning rate and discount factor. Using CoppeliaSim for virtual modeling, this work provides a high-fidelity platform that bridges the gap between numerical simulation and the implementation of physical systems. Through this systematic analysis, critical determinants of controller performance are elucidated, providing deep empirical insights and robust design principles for applying classical reinforcement learning to complex control problems in realistic physical simulations. [ABSTRACT FROM AUTHOR]
ISSN:	19929978