Skip to content
SPECIFICTION
Get listed

ChainForge

chainforge.ai

Free plan WebSelf-hosted
Visit chainforge.ai
Spec score 8.6 #3 of 28 in AI Prompt Generators

What it is

ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation, and experimentation. It can query multiple language models at once and compare results across prompts, models, and model settings. Responses can be scored with code-based or LLM-based evaluators, then plotted for review. Chat Turn nodes support continuing multiple conversations, templating chat messages, and handing a conversation from one model to another. The tool imports JSONL, XLSX, and CSV files and exports inspected response data as XLSX. Listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama. A local installation can add custom providers written as Python completion functions. The web version has a limited feature set; local use supports API keys from environment variables, Python evaluation code, and locally hosted Alpaca or Llama models via Dalai. Web users must set API keys each time they load the app, and the documentation says keys are not stored. ChainForge is available on the web or self-hosted, is free and open source, and is described as an open beta in active development.

Spec sheet

What it does
ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation.Read 8 Oct 2026 · chainforge.ai
Purpose
ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation and experimentation.Read 8 Oct 2026 · chainforge.ai
Support
The homepage directs users to submit GitHub issues or a Google Form for questions, comments or feature requests.Read 8 Oct 2026 · chainforge.ai
API key handling
The documentation says the web version does not store API keys in cookies, localStorage or the server, so users must set them each time they load ChainForge.Read 8 Oct 2026 · chainforge.ai
API keys
The documentation says ChainForge does not store API keys and that users must set them each time they load the web version.Read 8 Oct 2026 · chainforge.ai
Browser support
The installation guide lists Chrome, Firefox, Edge and Brave, and says the beta does not support other browsers.Read 8 Oct 2026 · chainforge.ai
Compare models and prompts
It can query multiple LLMs at once and compare response quality across prompt variations, models and model settings.Read 8 Oct 2026 · chainforge.ai
Compare prompts and models
It can query multiple LLMs at once and compare responses across prompt variations, models and model settings.Read 8 Oct 2026 · chainforge.ai
Conversation testing
Chat Turn nodes can continue multiple conversations, template chat messages and pass a conversation from one LLM to another.Read 8 Oct 2026 · chainforge.ai
Creator
The documentation says ChainForge was created by Ian Arawjo, who serves as its chief developer and maintainer.Read 8 Oct 2026 · chainforge.ai
Custom integrations
Locally installed ChainForge can add providers through Python completion functions loaded in the Custom Providers settings tab.Read 8 Oct 2026 · chainforge.ai
Custom providers
A locally installed version can call custom providers written as Python completion functions.Read 8 Oct 2026 · chainforge.ai
Data and exports
ChainForge can import JSONL, XLSX and CSV spreadsheets and export inspected response data as XLSX.Read 8 Oct 2026 · chainforge.ai
Encryption limitation
The documentation warns that encrypted flows and settings cannot be accessed if the user forgets the password.Read 8 Oct 2026 · chainforge.ai
Evaluate responses
Users can evaluate responses with code-based or LLM-based scorers and plot evaluation results.Read 8 Oct 2026 · chainforge.ai
Known security limitation
The documentation says forgetting the encryption password prevents access to saved flows or settings, and passwords are stored in hashed form and cannot be recovered.Read 8 Oct 2026 · chainforge.ai
Local encryption
When running locally, users can enable encryption for settings or all locally saved files with the secure-mode command options.Read 8 Oct 2026 · chainforge.ai
Local features
Local installation supports API keys from environment variables, Python evaluator nodes and Ollama-hosted models.Read 8 Oct 2026 · chainforge.ai
Project status
ChainForge is described as being in active development and provided as an open beta test.Read 8 Oct 2026 · chainforge.ai
Providers
The supported-models page lists OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama.Read 8 Oct 2026 · chainforge.ai
Web sharing
The web version has a Share button that creates a unique link to an LLM experiment.Read 8 Oct 2026 · chainforge.ai
Web version limit
The web version has a limited feature set; the full local version supports loading API keys from environment variables, Python evaluation code and locally hosted Alpaca or Llama models via Dalai.Read 8 Oct 2026 · chainforge.ai

Plans and prices

Free and open-source Free No plan limits stated Read 8 Oct 2026 · glassmanlab.seas.harvard.edu

Every ChainForge plan and price →

Strong points and limits

Strong points

  • Queries multiple LLMs in one experiment.
  • Supports code-based and LLM-based scoring.
  • Imports JSONL, XLSX, and CSV files.
  • Offers a self-hosted version with additional local features.
  • Free and open source.

Limits

  • The web version has a limited feature set.
  • Web users must re-enter API keys each time.
  • Forgetting an encryption password prevents access to saved flows or settings.
  • The beta does not support browsers beyond Chrome, Firefox, Edge, and Brave.

Who it suits

ChainForge suits people comparing prompts and model responses through visual experiments, including those who want to score or plot results. Local installation is more appropriate for users who need Python evaluation code or locally hosted models.

Verdict

ChainForge provides a visual route to comparing prompts and evaluating model responses, with broader capabilities in its local version. Its open-beta status, browser limits, and web API-key workflow are important considerations.

Where Specifiction places it

Alternatives to ChainForgeAll alternatives →