Safe and adaptive control of non-stationary stochastic systems via Lyapunov-constrained distributional reinforcement learning

Engineering notes

Simulations in chaotic and hyperchaotic systems showcase LG-DRL-ER’s superior performance in achieving stable, adaptive control compared to existing methods.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Abstract Non-stationary environments pose significant challenges for reinforcement learning (RL), particularly in safety-critical applications like robotics and energy systems, where adaptability, stability, and robustness to uncertainty are essential. This paper introduces Lyapunov-Guided Distributional Reinforcement Learning with Entropy Regularization (LG-DRL-ER), a novel framework designed to address these challenges in non-stationary Markov Decision Processes (MDPs). LG-DRL-ER integrates three key components: distributional RL to model the full return distribution, capturing reward uncertainty; Lyapunov stability constraints to ensure safe convergence to a target state; and entropy regularization to promote adaptive exploration. We present a rigorous stability analysis, supported by lemmas and theorems, demonstrating convergence in probability under dynamic conditions. We further establish finite-time convergence guarantees with explicit bounds on the convergence time. An adaptive parameter update rule ensures robustness to changing dynamics, validated through theoretical guarantees and empirical evaluations. Simulations in chaotic and hyperchaotic systems showcase LG-DRL-ER’s superior performance in achieving stable, adaptive control compared to existing methods. This framework advances safe RL by bridging data-driven learning with control-theoretic guarantees, offering significant implications for autonomous systems and dynamic environments.

Links and sources

• PDF from original source

Comments