Skip to content
SPECIFICTION
Get listed

Galileo vs LiveBench

Galileo

galileo.ai

LiveBench

livebench.ai

Side by side

FigureGalileoLiveBench
Price from$100/mo—
Free plan——
Free trial——
Open source——
WebYesYes
Windows—Yes
macOS—Yes
Linux—Yes
APIYes—
Self-hostedYesYes
Plans published30
Spec score · LLM Evaluation Tools7.8 (#11)7.2 (#22)
PurposeGalileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications.LiveBench is an LLM benchmark designed to limit test set contamination and use objective evaluation.
SupportThe Pro plan includes dedicated Slack support, and Enterprise includes 24/7 support via Slack, email, or phone.The repository invites model evaluation requests by GitHub issue or email at [email protected].

Plans

Galileo

  • Pro$100 a month
  • EnterprisePriced on request
  • FreePrice not published

Galileo pricing →

LiveBench

No plans published.

LiveBench pricing →

In their own words

Galileo

Galileo is an observability, evaluation and guardrail platform for generative AI and agentic applications. It provides ready-made evaluations for RAG, agents, safety and security, as well as custom evaluators. Observability tools analyze agent behavior to surface patterns and failure modes, while Galileo Experiments let teams evaluate datasets and prompts systematically and compare results. Evaluation scores can guide agent actions, tool access and escalation paths. Developers can use Python and TypeScript SDKs or the API, with integrations including CrewAI, Google ADK, LangChain and LangGraph, OpenAI, Microsoft Agent Framework, Strands Agents and Vercel AI SDK. Deployment options include SaaS, virtual private cloud and on-premises. The Free plan includes 5,000 traces per month, unlimited users and unlimited custom evaluations. Pro costs $100 a month, billed yearly, and includes 50,000 traces per month; pricing scales with trace volume. SSO supports OIDC and SAML only on a customer's own Galileo cluster, not the free version or hosted app.galileo.ai service.

Visit galileo.ai

LiveBench

LiveBench is an open-source benchmark for evaluating large language models, designed to limit test-set contamination and use objective scoring. Its current scope is 23 tasks across Reasoning, Coding, Agentic Coding, Mathematics, Data Analysis, Language, and Instruction Following, with a refresh every six months. The leaderboard presents overall and category scores, and model rows expand to show subtasks. Insight views cover quality versus cost, ranked cost, and category profiles; cost per successful task is calculated as (Σ cost ÷ Σ questions ÷ score) × 100 for the selected scope. Questions have verifiable ground-truth answers and can be scored automatically without an LLM judge. The public project provides benchmark code and links to data on Hugging Face, as well as a Python command-line evaluation pipeline. It supports OpenAI-compatible endpoints and implemented inference for Anthropic, Cohere, Mistral, Together, and Google models. Local inference is unmaintained. Agentic coding evaluations require Docker, and task images may take up to 150 GB to store.

Visit livebench.ai