Build on what the community has made.
A carefully selected starting set of public frameworks and benchmarks. Always review original documentation, current licenses, and implementation details before adopting a project.
Verifiers
An open-source library for building environments for evaluating and training language models.
Visit official resource ↗NVIDIA NeMo Gym
Environment infrastructure with task datasets, agent harnesses, verification, and state.
Visit official resource ↗BrowserGym
Extensible framework spanning browser tasks and benchmark integrations.
Visit official resource ↗SWE-bench
Benchmark family for software engineering agents working against real repository issues.
Visit official resource ↗OSWorld
Benchmark for agents working in real desktop computer environments.
Visit official resource ↗Gymnasium
Common API for single-agent reinforcement learning environments.
Visit official resource ↗PettingZoo
Environment interfaces for multi-agent reinforcement learning research.
Visit official resource ↗τ-bench
Research benchmark around tool-using conversational agents and policy-constrained tasks.
Visit official resource ↗