ChainForge vs GEPA

ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation, and experimentation. It can query multiple language models at once and compare results across prompts, models, and model settings. Responses can be scored with code-based or LLM-based evaluators, then plotted for review. Chat Turn nodes support continuing multiple conversations, templating chat messages, and handing a conversation from one model to another. The tool imports JSONL, XLSX, and CSV files and exports inspected response data as XLSX. Listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama. A local installation can add custom providers written as Python completion functions. The web version has a limited feature set; local use supports API keys from environment variables, Python evaluation code, and locally hosted Alpaca or Llama models via Dalai. Web users must set API keys each time they load the app, and the documentation says keys are not stored. ChainForge is available on the web or self-hosted, is free and open source, and is described as an open beta in active development.

GEPA is a free, open-source Python package for optimizing prompts, code, configurations, agent architectures, policies, and other text-based artifacts. Its optimize_anything API connects to any system through an evaluator you write, so DSPy or another framework is not required. GEPA examines execution traces and evaluator feedback, uses language-model reflection to diagnose problems and propose targeted mutations, then selects candidates with Pareto-aware criteria. It can optimize API-only GPT, Claude, and Gemini models without model-weight access. Human-readable traces explain changes and support agent debugging. Built-in adapters cover Default, Confidence, DSPy Full Program, Generic RAG, MCP, TerminalBench, AnyMaths, and LangChain, with integrations including MLflow, Comet ML Opik, Pydantic AI, and cookbook projects. You can limit metric calls and set several stopping conditions. GEPA works through API or self-hosted deployment, installs with pip, and uses the MIT license.