ByteDance Seed
首页模型博客&论文Seed Edge加入我们
体验豆包AI 体验中心
首页模型博客&论文Seed Edge加入我们
体验豆包AI 体验中心

2025-05-20

DAPO: An Open-Source LLM Reinforcement Learning System at Scale

Download PDF
上一篇下一篇

摘要

Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning. However, key technical details of state-of-the-art reasoning LLMs are concealed (such as in OpenAI o1 blog and DeepSeek R1 technical report), thus the community still struggles to reproduce their RL training results. We propose the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm, and fully open-source a state-of-the-art large-scale RL system that achieves 50 points on AIME 2024 using Qwen2.5-32B base model. Unlike previous works that withhold training details, we introduce four key techniques of our algorithm that make large-scale LLM RL a success. In addition, we open-source our training code, which is built on the verl framework, along with a carefully curated and processed dataset. These components of our open-source system enhance reproducibility and support future research in large-scale LLM RL.

作者

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, Mingxuan Wang

期刊/会议

NeurIPS 2025

追求智能上限,创造社会价值
字节跳动 Seed 招聘二维码
欢迎加入字节跳动 Seed
Copyright© 2026 Bytedance Seed
网站声明联系我们 : seed.feedback@bytedance.com
追求智能上限,创造社会价值
字节跳动 Seed 招聘二维码
欢迎加入字节跳动 Seed
联系我们 : seed.feedback@bytedance.com
Copyright© 2026 Bytedance Seed网站声明联系我们 : seed.feedback@bytedance.com