Autonomous driving paper index
Distribution-aware reincarnating reinforcement learning mitigates dual distribution shifts in teacher-guided offline-to-online learning
One-line summary
Offline pretraining followed by online fine-tuning is a classical paradigm in reinforcement learning.
Engineering notes
Experiments on six Atari tasks show that the proposed method achieves strong overall performance, improves Q-value conservatism, and yields smoother offline-to-online transfer.
Chinese explanation / 中文解读
中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。
Original abstract
Offline pretraining followed by online fine-tuning is a classical paradigm in reinforcement learning. When a pretrained teacher is available, this paradigm can be extended from generic initialization to efficient teacher-guided learning. Reincarnating Reinforcement Learning (RRL) is a representative framework in this setting. However, in teacher-guided offline-to-online learning, the transferred teacher knowledge may become unreliable under distribution shift. In particular, the offline stage suffers from out-of-distribution (OOD) action extrapolation on teacher replay, while the online stage suffers from replay-distribution shift during the transition from teacher-generated to student-generated experience. We address these issues with Distribution-Aware Reincarnating Reinforcement Learning, which combines teacher-guided conservative learning for reliable offline value reuse and an adaptive balanced replay buffer for stable online replay transition. Experiments on six Atari tasks show that the proposed method achieves strong overall performance, improves Q-value conservatism, and yields smoother offline-to-online transfer.
Links and sources
Need this topic turned into a technical roadmap?
Full Self Driving can prepare a custom autonomous driving literature review, code map, dataset map, and B2B technology assessment.
Request B2B research
Comments