S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?

ByteDance Seed (with M-A-P and TokenWave.AI) asks whether an agent can actively test its own behavior, judge the resulting experience, and use it to improve future decisions — rather than being evaluated as a fixed policy. S³Gym couples three capabilities (Self-Testing, Self-Judging, Self-Improvement) across seven text-based games with executable environment verifiers, separating permissive exploration from strict held-out evaluation, and compares three pathways for incorporating experience: direct history ICL, score-conditioned summary memory, and parameter training.

Self-improvement proves neither automatic nor uniform: summaries help when experience compresses into reusable strategic rules but underperform raw history when success depends on precise, state-contingent information; parameter training yields substantial gains on some tasks but unstable improvement and severe negative transfer on others. Recognizing successful actions is insufficient — agents must transform feedback into executable, transferable policies. The experience-integration leg of the Seed team's Self-Developing Agents trilogy, alongside Aspire (goal formation) and HarnessDev (system integration).