Julius AI vs Kiln

Julius AI has no write-up on Specifiction yet; its figures are above.

Kiln is an AI workbench for teams building and improving AI systems, with evaluation, fine-tuning, and optimization tools. Users can describe an evaluation goal and generate prompts and synthetic datasets, then compare judge results with human ratings. Kiln supports composing agents and subagents, inspecting tool calls, and evaluating each level of an agent hierarchy. It supports MCP servers and providers including AWS Bedrock, Azure OpenAI, Vertex AI, OpenAI, Anthropic, Fireworks, Gemini, Ollama, and OpenAI-compatible endpoints. Datasets, traces, and evaluations sync to Git. Prompts, outputs, ratings, and fine-tuning jobs are stored as JSON files on the user's filesystem, while provider API calls go directly from the user's machine to the chosen provider. Kiln says it can run fully offline with Ollama, vLLM, Llama.cpp, or another compatible local server. The free Individual plan includes a Python library, source-available desktop app, Git-synced local datasets, rate-limited standard models, and community support.