Ai2's evaluation workbench extending OLMES: modular task/suite/harness abstractions, sandboxed tool-use evaluation routing, and first-class support for agentic and multi-turn evals. Apache-2.0. The eval infrastructure behind the OLMo and Tmax lines.

Library

Language Python
License Apache 2.0
evalinfrastructurelibrary

Related