PrivateGPT

PrivateGPT is an open-source API layer for developers building private AI applications around models they operate or select. It is strongest when document retrieval, structured data and tool use need to sit behind one developer-facing interface; it is not a model host.

Overview

PrivateGPT runs as a local server and can be installed as a Python package, from source or with Docker. It connects to a separate OpenAI-compatible inference server, with documented options including Ollama, LM Studio, llama.cpp and vLLM. That flexibility suits teams with an existing model setup, but leaves them responsible for deploying and maintaining inference as well as PrivateGPT.

The project is free and open source. Application data, including ingested documents, vector-store data, models and caches, is stored under a local home directory by default. Developers can also use a remote compatible server, so local storage does not by itself keep inference on the same machine. Optional observability can trace LLM calls, embedding requests and retrieval through the console, Arize Phoenix or Opik.

Key features

The Messages API supports streaming, asynchronous processing and token counting. For developers building applications rather than a single prompt interface, those controls support more flexible request handling. File and artifact ingestion feed retrieval-augmented generation, with citations that help users trace answers to source material.

PrivateGPT also supports structured access to databases and tabular data such as CSVs. Documented tools extend to web search and extraction, code execution, custom tools and MCP connectors, with MCP usage available through the Messages API. Its compatibility table marks skills basic or in progress, and does not support prompt caching or OAuth and organizations; those gaps matter for teams that need mature skills or built-in organizational access controls.

A browser workbench at /ui provides a place to test, demonstrate and inspect API requests. It is explicitly a local demonstrator: chats and settings are stored in browser localStorage, and it has no access control. It is useful for development, not a secure shared workspace.

Pricing

PlanPriceIncludesPrivateGPTfreeOpen-source API layer; self-hosted; connects to a separate OpenAI-compatible inference serverThere is no paid plan to weigh against the free self-hosted option. The cost is operational rather than a subscription: developers need to provide and maintain a compatible inference server.

Platforms

PrivateGPT supports API, Linux, macOS, self-hosted, web and Windows. Package installation instructions cover macOS, Linux and Windows, and Docker is also documented. Package users need Python 3.11; Python 3.10 and Python 3.12 or later are unsupported. The browser workbench is available through the local server.

Who it's for

PrivateGPT is a good fit for developers who want an open-source application layer for document retrieval, data access and tool workflows on infrastructure they control. Its API breadth is useful when a team already has a compatible inference server and can manage the additional deployment components. It is a weaker fit for users seeking a self-contained model service, a protected multi-user workbench or supported package installation on Python versions outside 3.11.

Pros and cons

Pros

Free, open-source and self-hosted, with default local storage for application data.

Messages API combines streaming, asynchronous processing and token counting with ingestion, retrieval and citations.

Connections to OpenAI-compatible servers let teams choose among documented providers such as Ollama, LM Studio, llama.cpp and vLLM.

Database and CSV access, MCP, web tools and code execution broaden it beyond document-only retrieval.

Cons

Inference runs on a separate server, adding deployment and maintenance work.

Python package installation requires 3.11 and excludes both 3.10 and 3.12 or later.

The workbench has no access control and stores chats and settings in browser localStorage.

Skills are marked basic or in progress, while prompt caching and OAuth or organizations are unsupported.

Alternatives

Retrieval-Augmented Generation Tools is a useful starting point for comparing products in this category.

AnythingLLM is worth considering if Android support or a freemium option matters; its self-hosted Docker deployment is open source, while its Basic plan is $50 a month.

Flowise may suit teams wanting a freemium workflow builder: its free plan allows 2 Flows & Assistants, 100 Predictions per month and 5MB storage.

RAGFlow is an option for a hosted freemium setup, with a free plan limited to 5 apps, 1 team member, 0.1 GB dataset storage and 500 credits per month, and no API key.

txtai is another free, open-source self-hosted option, installable via pip or Docker.

Haystack Enterprise Platform offers a free Studio tier with one workspace, one user, 100 pipeline hours and cloud deployment.

I3K RAG Enterprise has a free AGPL-3.0 community edition for self-hosting on Linux x86_64, ARM64 or Windows.

Google Cloud Agent Evaluation is a paid evaluation service with usage-based charges for computation-based metrics; model-based metrics are charged separately.

Haystack is a free, open-source Python framework for retrieval-augmented generation.

Verdict

Choose PrivateGPT if you want a free, open-source API layer for document and data workflows and are prepared to run a compatible inference server yourself. Its breadth across retrieval, citations and tools is the reason to pick it; look elsewhere if you need inference bundled in, a secure shared workbench or a less constrained Python package requirement.