SAO: Single-Rollout Asynchronous Optimization

"Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning" — the RL recipe behind GLM-5.2 (750B-A40B). From Tsinghua authors working at Z.AI, with Jie Tang and Yuxiao Dong as senior authors. SAO replaces group-wise sampling (the GRPO pattern) with a single rollout per prompt for asynchronous pipelines, stabilized by three mechanisms: direct double-sided importance sampling with strict token-level clipping that masks divergent updates entirely, single-rollout sampling to cut off-policy drift on agentic tasks, and enhanced value-model training (faster value than policy updates, frozen attention during value training, skip-observation GAE for multi-turn trajectories). Reported gains over GRPO baselines include AIME2025 97.3% (vs. 85.0% SFT) and SWE-bench Verified 29.8% (vs. 23%), plus online adaptation under dynamic reward shifts. Notable as a rare disclosure of the production RL method behind a frontier open-weight model.