Rhesis AI vs Vellum
Rhesis AI
rhesis.aiVellum
vellum.aiSide by side
| Figure | Rhesis AI | Vellum |
|---|---|---|
| Price from | Free | $30/mo |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | Yes |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | Yes | — |
| iOS | — | Yes |
| Android | — | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 2 | 9 |
| Spec score · AI LLM Evaluation Tools | 7.5 (#23) | 8.6 (#9) |
| Headquarters | Potsdam, Germany | United States |
| Product | Rhesis is an open-source collaboration layer that brings domain-expert knowledge into AI agent development. | Vellum describes itself as a personal AI assistant that can read files, manage calendars, send email, run code, and control a computer. |
| Support | The getting-started page points users to in-app support, documentation, a support email, GitHub, and Discord. | Vellum lists support contact as [email protected]. |
Plans
Vellum
- BaseFree
- FreeFree
- Mighty$30 a month
- Mighty$30 a month
- Custom$50 a month
- Super$100 a month
- Super$100 a month
- Ultra$200 a month
- Ultra$200 a month
In their own words
Rhesis AI
Rhesis AI is an open-source collaboration layer for developing and evaluating AI agents. Domain experts, engineers, and product managers can work in a shared space to review test cases and agent versions, annotate them, and record expectations. Teams can generate test sets from application requirements or constraints, including single-turn prompts and multi-turn conversations, then run them against an application endpoint and evaluate responses using configured metrics. Results are saved in test runs. Rhesis Tracing uses OpenTelemetry to capture language-model calls, tool invocations, and retrieval operations in test or production mode. Evaluation options include offline evaluation, online trace metrics, LLM-as-a-judge, custom metrics, and adversarial red-teaming. Model providers connect through LiteLLM, including cloud and local options such as OpenAI, Anthropic, Google Gemini, Mistral, Ollama, and vLLM. Teams can use DeepEval and Ragas metrics or import Garak probes. A REST API supports test sets, runs, results, and trace inspection for integrations and CI/CD. Rhesis offers managed, Docker-local, and self-hosted use; it says it is developed in Germany and hosted in the EU. Both listed free editions cost $0.
Vellum
Vellum is a personal AI assistant that can read files, manage calendars, send email, run code, and control a computer. It saves preferences and context in its workspace for later conversations, and users can create, install, and manage skills. Listed OAuth services include Discord, GitHub, Google services, HubSpot, Linear, Twitter (X), Asana, Notion, Microsoft Outlook, and Todoist. Vellum says OAuth tokens stay in a credential vault rather than being exposed to the language model. Sensitive actions prompt for approval, and users can set trust rules for standing permissions. By default, conversation context is sent to Anthropic to generate responses; OpenAI, Google, OpenRouter, Fireworks, and local Ollama are listed alternatives. An internet connection is required. Vellum Cloud runs assistants in isolated environments with private storage and 24/7 availability. The Free plan includes first tasks and pay-as-you-go credits. Mighty costs $30 a month, Super $100 a month, and Ultra $200 a month. There is no shipped Linux desktop client, but Linux users can use the web app, Chrome extension, or self-host the runtime.