Promptotype vs PromptWizard

Promptotype is a web-based platform for developing and evaluating structured tasks for language models. Its playground lets users build prompt templates, configure models, and work with function calling. Test queries can include expected JSON schemas or values; a model query can fill in an expected result. Users can run prompts against collections, from smaller development sets to larger checks for production readiness. The platform also provides run and test history, scheduled automated tests with email summaries, and automatic fine-tuned model creation from query collections. A JSON Schema Editor supports schemas compatible with the OpenAI API’s json_schema response_format. The free plan is free. Basic costs $6 a month and adds libraries for templates, configurations, and queries, plus collection testing. Professional costs $10 a month and includes 60 days of history, fine-tuning, and one weekly scheduled run. Team features are contact-based. The maker is Baionic, LLC, which lists a New York, New York mailing address.

PromptWizard is a free, MIT-licensed open-source framework for automated prompt and example optimization. It generates prompt variations, scores them, critiques their outcomes, and refines instructions over repeated rounds. It can also optimize in-context examples and synthesize examples suited to a task. The described workflows cover prompt optimization without examples, with generated examples, or with training data. Listed datasets include GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL samples with question and answer fields, and require task-specific answer extraction and evaluation functions, configuration, and data files. PromptWizard can generate chain-of-thought reasoning for in-context examples, with an option to turn that off to reduce prompt length. Installation uses the GitHub repository as a Python package in development mode, with instructions for Windows, macOS, and Linux. Model access setup supports OpenAI API keys and Azure OpenAI endpoints, so access requires credentials for those services. The README reports average optimization times of around 20–30 minutes in experiments on the listed datasets, with timing dependent on the dataset, and notes that human supervision can help tune outputs.