Argilla vs Prodigy

Argilla is a free collaboration tool for AI engineers and domain experts building datasets with human feedback. It supports NLP, LLM, and multimodal projects, including classification, named entity recognition, retrieval-augmented generation, and preference tuning. Contributors can provide labels, ratings, rankings, and written responses; annotation tools include filters, AI feedback suggestions, and semantic search. The system handles text and images, while custom fields can represent audio, video, or other content rendered as base64 or HTML. It is compatible with Hugging Face and spaCy, and tutorials cover SetFit and LlamaIndex. Deployment options include Hugging Face Spaces or a local machine or server using Docker Compose. A Python SDK connects to an Argilla server with an API URL and key, and the server exposes REST API documentation. Datasets and records can be moved to or from Python, local disk, and the Hugging Face Hub. Argilla supports OAuth2 through Hugging Face, GitHub, and Google by default. It does not train models; on Hugging Face Spaces, data on ephemeral free storage is lost when the Space restarts.

Prodigy is a downloadable developer tool and Python library for creating training and evaluation data for machine-learning models. Its built-in recipes cover named entity recognition, text classification, computer vision, audio and video annotation, transcription, and model training and evaluation. A customizable web application offers 23 interfaces for work with text, images, audio, video, relations, multiple choice, HTML and conflict resolution. Users can define workflows with Python recipes and create interfaces using HTML, CSS and JavaScript. Prodigy supports spaCy, plugins for Hugging Face models and integrations with major LLM API providers. Data can be stored in SQLite, MySQL or PostgreSQL and exported to JSON. It runs on the user's own machines without connecting to the maker's or third-party servers, including on air-gapped machines. The standard Python web server and app can be deployed in the cloud or on-premise. Personal licensing costs $390, paid once, excluding tax; Company licensing costs $490, paid once per seat in packs of five, excluding tax.