Balanced Sampling and Reusing Imaginary Data for World Models in Reinforcement Learning

doi:10.1109/TAI.2025.3592174

科研成果详情

题名	Balanced Sampling and Reusing Imaginary Data for World Models in Reinforcement Learning
作者	Wang，Qianyu 1; Wei，Xuekai 1; Yan，Jielu 1; Leong，Hou H.2; Pu，Huayan 3; Luo，Jun 3; Jia，Weijia 4,5; Zhou，Mingliang 1
发表日期	2025
发表期刊	IEEE Transactions on Artificial Intelligence
摘要	Deep reinforcement learning (DRL) has shown significant success in domains such as computer vision and robot control. However, DRL agents often suffer from low sample efficiency, limiting their practical applicability in industrial settings. Recent advances in model-based DRL, particularly model-based approaches, have sought to address this issue by leveraging imaginary data to improve decision-making and sampling efficiency. Despite their promise, these methods face challenges such as overreliance on early experiences in the replay buffer and under-utilization of imaginary data, which can lead to overfitting and suboptimal policy optimization. To overcome these limitations, we propose a novel reinforcement learning framework, balanced sampling and reusing imaginary data (BSRID), which introduces two key innovations: (1) a balanced sampling (BS) mechanism that ensures uniform sampling rates to mitigate bias toward early experiences and (2) a reusing imaginary data (RID) strategy that enhances policy optimization by increasing update frequency and maximizing the utility of imaginary data. The experimental results on the Atari 100k benchmark demonstrate that BSRID significantly improves sample efficiency and achieves state-of-the-art performance. This work provides a robust and efficient solution for DRL applications in scenarios requiring high sample efficiency and reliable decision making.
关键词	Deep reinforcement learning imaginary data sample efficiency world model
DOI	10.1109/TAI.2025.3592174
URL	查看来源
语种	英语English
Scopus入藏号	2-s2.0-105012146502
引用统计
文献类型	期刊论文
条目标识符	https://repository.uic.edu.cn/handle/39GCC9TT/13738
专题	北师香港浸会大学
通讯作者	Wei，Xuekai; Yan，Jielu
作者单位	1.Chongqing University,School of Computer Science,Chongqing,400044,China 2.University of Macau,State Key Laboratory of Internet of Things for Smart City,Centre for Data Science,Department of Computer and Information Science,Macao 3.Chongqing University,State Key Laboratory of Mechanical Transmissions,Chongqing,400044,China 4.BNU-UIC Institute of Artificial Intelligence and Future Networks,Beijing Normal University,Zhuhai,Guangdong,519087,China 5.Guangdong Key Laboratory of AI Multi-Modal Data Processing,BNU-HKBU United International College,Zhuhai,Guangdong,519087,China
推荐引用方式 GB/T 7714	Wang，Qianyu,Wei，Xuekai,Yan，Jieluet al. Balanced Sampling and Reusing Imaginary Data for World Models in Reinforcement Learning[J]. IEEE Transactions on Artificial Intelligence, 2025.
APA	Wang，Qianyu., Wei，Xuekai., Yan，Jielu., Leong，Hou H., Pu，Huayan., .. & Zhou，Mingliang. (2025). Balanced Sampling and Reusing Imaginary Data for World Models in Reinforcement Learning. IEEE Transactions on Artificial Intelligence.
MLA	Wang，Qianyu,et al."Balanced Sampling and Reusing Imaginary Data for World Models in Reinforcement Learning". IEEE Transactions on Artificial Intelligence (2025).