Open-Reasoner-Zero: Scaling Up RL on the Base Model

StepFun and Tsinghua (first author Jingcheng Hu; senior author Harry Shum) delivered the first fully open implementation of large-scale reasoning-oriented RL directly on a base model, weeks after DeepSeek-R1-Zero. The recipe is deliberately minimalist: vanilla PPO with GAE (λ=1, γ=1), simple rule-based rewards, and no KL regularization — yet it reproduces the R1-Zero scaling phenomenon in which both benchmark performance and response length grow with training, and from the same Qwen2.5-32B base outperforms DeepSeek-R1-Zero-Qwen-32B on AIME 2024, MATH500 and GPQA Diamond.

In the RL-scaling literature it anchors the open, sequential-steps end: IsoCompute cites it (with DAPO) as work that scales sequential updates to reward convergence without framing itself as a scaling-law study. Training code, data and weights are fully released.