Exgentic

Spec sheet

Purpose

Exgentic is an open agent leaderboard that evaluates general purpose AI agents across diverse domains.Read 9 Oct 2026 · exgentic.ai

Product

Exgentic is an open agent leaderboard that compares general purpose AI agents across diverse benchmarks.Read 4 Oct 2026 · exgentic.ai

Integrations

The README documents API credentials for OpenAI and Anthropic models and support for Hugging Face models and Jobs.Read 9 Oct 2026 · github.com

Support

The project directs users to open a GitHub issue for questions and support.Read 9 Oct 2026 · github.com

License

The framework is distributed under the Apache License 2.0.Read 9 Oct 2026 · github.com

Agents

Listed agents include LiteLLM Tool Calling, SmolAgents, OpenAI MCP, Claude Code, Codex CLI, and Gemini CLI.Read 4 Oct 2026 · github.com

Available agents

The listed agents include LiteLLM Tool Calling, HuggingFace SmolAgents, OpenAI MCP, Claude Code, Codex CLI, and Gemini CLI.Read 9 Oct 2026 · github.com

Available benchmarks

The framework lists tau2, appworld, browsecompplus, swebench, hotpotqa, gsm8k, and bfcl benchmarks.Read 9 Oct 2026 · github.com

Benchmark handling

Exgentic says it does not modify benchmarks or agent implementations, though it may adapt interfaces to its unified protocol.Read 4 Oct 2026 · exgentic.ai

Benchmarks

The leaderboard covers personal assistance, customer service, technical support, deep research, and software engineering benchmarks.Read 9 Oct 2026 · exgentic.ai

Evaluation approach

Exgentic says it avoids prompt optimization and reports results on 100 sampled tasks per benchmark.Read 9 Oct 2026 · exgentic.ai

Evaluation domains

The leaderboard covers personal assistance, customer service, technical support, deep research, and software engineering tasks.Read 4 Oct 2026 · exgentic.ai

Evaluation framework

Its framework provides a consistent interface for evaluating agents on benchmarks to compare performance and reproduce results.Read 4 Oct 2026 · github.com

Framework

Exgentic provides a consistent interface for testing agents across benchmarks and domains to compare performance and reproduce results.Read 9 Oct 2026 · github.com

Installation

The project documents installation as a command line tool with uv and use as a Python library with uv or pip.Read 4 Oct 2026 · github.com

Isolation

The repository documents a Docker runner for full container isolation, requiring Docker to be installed and running.Read 4 Oct 2026 · github.com

License and support

The repository states that Exgentic is licensed under Apache License 2.0 and directs support questions to GitHub issues.Read 4 Oct 2026 · github.com

Local use

The README documents installation as a command-line tool or Python library and includes an evaluation dashboard command.Read 9 Oct 2026 · github.com

Methodology

Exgentic says it avoids prompt optimization and reports results on 100 sampled tasks per benchmark.Read 4 Oct 2026 · exgentic.ai

Model integrations

The quick start documents OpenAI and Anthropic API credentials, and the repository also describes support for Hugging Face models or evaluations on Hugging Face Jobs.Read 4 Oct 2026 · github.com

Open results

The evaluation pipeline, agent implementations, configuration, and results are available for reproducing evaluations.Read 9 Oct 2026 · github.com

Reproducibility

The evaluation pipeline, agent implementations, and configuration are available in the repository; the project notes results may vary with model versions or nondeterministic outputs.Read 4 Oct 2026 · exgentic.ai

Run limits

Evaluation sessions default to limits of 100 steps and 100 actions, and stop when either limit is reached.Read 9 Oct 2026 · github.com

Security and isolation

The README documents a Docker runner for full container isolation when running evaluations.Read 9 Oct 2026 · github.com

Use cases

The repository identifies agent builders, researchers and component developers, and benchmark builders as intended users.Read 4 Oct 2026 · github.com

Who it is for

The framework is intended for the general audience, agent builders, researchers and component developers, and benchmark builders.Read 9 Oct 2026 · github.com

Where Specifiction places it

• AI Agent Evaluation Tools#17 of 287.8