Harness-of-Harness: Multi-Day Autonomous Software Development
paperYour notes
Shanghai AI Laboratory's harness that orchestrates existing coding-agent harnesses into iterative planning–coding–testing loops, aiming at autonomous software development: turning high-level requirements into complete, usable software with no human in the loop. HoH balances repair against capability growth, scopes development into small verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows.
Across GameCraft-Bench, FrontierSWE, and ProgramBench with three harness–model pairs (Codex + GPT-5.5 high, OpenCode + DeepSeek-V4-Pro, Pi + MiniMax-M3), HoH delivers an average relative gain of 52.25% (max 82.86%) after three iterations — e.g. Codex + GPT-5.5 rises from 49.58 to 71.52 on GameCraft-Bench and 44% to 71% FrontierSWE dominance. The headline demo: a multi-day deployment of 70+ iterations autonomously built Fusepoint, a human-playable narrative first-person shooter — coherent storyline, fully implemented mechanics, polished visuals, integrated audio — from an empty workspace containing only a product-requirements document. Code released under MIT.