Agent Lightning v1.0: Towards Harnessed Agentic RL

Microsoft Research's Agent Lightning v1.0 names and systematizes harnessed agentic RL: post-training in which the deploy-time agent harness (mini-SWE-agent, OpenHands, Claude Code, OpenClaw, Hermes…) owns the environment-interaction loop while the trainer sees only a stream of LLM request–response pairs through an endpoint proxy. The original Agent Lightning (arXiv 2508.03680, August 2025) introduced this disaggregated proxy architecture, since adopted by verl Uni-Agent, AReaL 2.0, slime v0.3.0 and Polar. The v1.0 paper gives the first systematic treatment of the problems the proxy view creates — retokenization of harness-side text, merging per-call samples into training rollouts, advantage calculation, loss normalization and training-backend scheduling — and shows that leaving them underspecified yields ineffective or unstable training. The framework is ~3,500 lines of code: an API Gateway that stores rollouts and forwards LLM calls, a Rollout Controller that runs agents on Kubernetes or a local process pool, and a collocated async RL mode that time-shares one GPU pool between rollout and weight update without exposing phase switches to the harness.

Validated on instruction-following, search and coding agents. For coding, where existing RL frameworks lack data and complete training scripts, the release includes a SWE-smith data-cleaning pipeline and reproducible scripts: RL alone with 6K training samples lifts Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4% (+14.6 points). Co-first authors from Fudan and Zhejiang University; one co-author at the University of Edinburgh. MIT-licensed; the GitHub repo has ~17.8K stars.