ChainForge vs PromptPerfect

ChainForge is an open-source visual programming environment for prompt engineering, LLM evaluation, and experimentation. It can query multiple language models at once and compare results across prompts, models, and model settings. Responses can be scored with code-based or LLM-based evaluators, then plotted for review. Chat Turn nodes support continuing multiple conversations, templating chat messages, and handing a conversation from one model to another. The tool imports JSONL, XLSX, and CSV files and exports inspected response data as XLSX. Listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock, and Ollama. A local installation can add custom providers written as Python completion functions. The web version has a limited feature set; local use supports API keys from environment variables, Python evaluation code, and locally hosted Alpaca or Llama models via Dalai. Web users must set API keys each time they load the app, and the documentation says keys are not stored. ChainForge is available on the web or self-hosted, is free and open source, and is described as an open beta in active development.

PromptPerfect is an open-source prompt optimizer from Beagle, available as a web app, Chrome extension, and API. In the web app, users submit a draft, choose an optimization mode, and receive rewritten text with an explanation of the changes. Modes include better, specific, cot, developer, research, beginner, product, and marketing. The Chrome extension is described as working on sites including ChatGPT and Gmail. Its API accepts a draft prompt and returns optimized text, an explanation, changes, the original text, provider, and model. Gemini, OpenAI, and Anthropic providers are supported. API keys can be supplied in the request body or as a Bearer token, with the Bearer token taking precedence. The homepage says keys stay in the browser and the project uses the MIT License. The Free plan costs nothing. Hosted Gemini anonymous callers share configured server-key quota.