Argilla vs BRAT

Argilla is a free collaboration tool for AI engineers and domain experts building datasets with human feedback. It supports NLP, LLM, and multimodal projects, including classification, named entity recognition, retrieval-augmented generation, and preference tuning. Contributors can provide labels, ratings, rankings, and written responses; annotation tools include filters, AI feedback suggestions, and semantic search. The system handles text and images, while custom fields can represent audio, video, or other content rendered as base64 or HTML. It is compatible with Hugging Face and spaCy, and tutorials cover SetFit and LlamaIndex. Deployment options include Hugging Face Spaces or a local machine or server using Docker Compose. A Python SDK connects to an Argilla server with an API URL and key, and the server exposes REST API documentation. Datasets and records can be moved to or from Python, local disk, and the Hugging Face Hub. Argilla supports OAuth2 through Hugging Face, GitHub, and Google by default. It does not train models; on Hugging Face Spaces, data on ephemeral free storage is lost when the Space restarts.

brat is a free, open-source tool for structured text annotation, with annotations represented in forms computers can process. It supports text spans, binary relations, n-ary associations, attributes, normalization annotations, and freeform notes. Teams can configure categories, types, and constraints for their tasks, and multiple annotators can work on the same document or collection while seeing one another’s edits nearly as they happen. Normalization can link annotations to external resources, and brat can incorporate output from automatic annotation services available as web services. Its server and client support Unicode across nearly 100 scripts. The tool checks edits against configured constraints and searches documents or collections, including keyword-in-context concordancing. Annotations export in brat stand-off format; visualizations can be saved as SVG, PNG, PDF, or EPS. The free v1.3 release is self-hosted under the MIT License. New server installations assume a UNIX-like environment; Windows users can run brat in a UNIX-like virtual machine. The experimental standalone server is not security-hardened and is not recommended for sensitive data or systems accessible from the internet.