Skip to content
SPECIFICTION
Get listed

PromptMetrics

github.com

WebWindowsmacOSLinuxAPISelf-hosted
Visit github.com
Spec score 7.5 #21 of 26 in Prompt Management Software

What it is

PromptMetrics is a free, open-source, self-hosted prompt registry for managing prompt versions and observing LLM activity. Prompts can be stored, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git. The system logs request metadata including model, tokens, latency, cost, and custom tags, and exports data to stdout JSON or OpenTelemetry. It also tracks agent traces, spans, workflow runs, and prompt versions labeled by environment. Evaluation suites can run against datasets with budget tracking, while A/B tests compare prompt versions and can promote a winner. A compliance scanner checks prompts for PII, API keys, URLs, and IP addresses, then assigns risk scores. Its playground can proxy calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI. Prompt content can use filesystem, GitHub, or S3 storage; metadata can use SQLite or PostgreSQL. Interfaces include a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs. Installation options include npm, Docker Compose, and building from source. The repository identifies the license as MIT.

Spec sheet

What it does
PromptMetrics is a lightweight, self-hosted prompt registry for versioning prompts and tracking LLM activity.Read 8 Oct 2026 · github.com
Purpose
PromptMetrics is a lightweight, self-hosted prompt registry with Git-backed versioning, metadata logging, and evaluations for LLM observability.Read 8 Oct 2026 · github.com
Deployment
The README provides installation options using npm, Docker Compose, or source, and describes the product as self-hosted.Read 8 Oct 2026 · github.com
License
The repository identifies the project as MIT licensed.Read 8 Oct 2026 · github.com
Headquarters
Berlin, GermanyRead 28 Sep 2026 · github.com
Agent telemetry
It tracks agent traces and spans, workflow runs, and prompt versions tagged with environment labels.Read 8 Oct 2026 · github.com
Compliance scanning
Its compliance engine scans prompts for PII, API keys, URLs, and IP addresses and assigns risk scores.Read 8 Oct 2026 · github.com
Dashboard
An optional Next.js dashboard provides charts and views for prompts, logs, traces, evaluations, A/B tests, compliance, and the playground.Read 8 Oct 2026 · github.com
Dashboard key handling
The dashboard authenticates through a BFF proxy so API keys are not stored in browser localStorage.Read 8 Oct 2026 · github.com
Evaluations and A/B tests
Evaluation suites can be run against datasets with budget tracking, and A/B tests compare prompt versions and can promote a winner.Read 8 Oct 2026 · github.com
Evaluations and testing
It supports evaluation suites, dataset-backed evaluation runs, and statistical A/B comparisons with winner promotion.Read 8 Oct 2026 · github.com
Installation
The README documents installation through npm, Docker Compose, or building from source.Read 8 Oct 2026 · github.com
Interfaces and SDKs
The project provides a REST API, CLI, optional Next.js dashboard, and Node.js and Python SDKs.Read 8 Oct 2026 · github.com
LLM providers
The playground can proxy chat and completion calls for OpenAI, Anthropic, Cohere, Ollama, and Azure OpenAI.Read 8 Oct 2026 · github.com
Logging
Structured logs can capture model, token, latency, cost, and custom-tag metadata and export to stdout JSON or OpenTelemetry.Read 8 Oct 2026 · github.com
Maker and company details
The repository says it is maintained by the PromptMetrics team; it does not state a headquarters or founding date.Read 8 Oct 2026 · github.com
Observability
It logs LLM request metadata such as model, tokens, latency, cost, and custom tags, and can export data through OpenTelemetry.Read 8 Oct 2026 · github.com
Prompt versioning
Prompts can be stored, versioned, retrieved, and rolled back through a REST API or CLI, with changes recorded in Git.Read 8 Oct 2026 · github.com
Rate limit
The README lists a default limit of 300 requests per 60-second window per API key, configurable through environment variables.Read 8 Oct 2026 · github.com
SDKs
The project lists Node.js and Python SDKs for programmatic access.Read 8 Oct 2026 · github.com
Security controls
API keys are HMAC-SHA256 hashed, support scoped permissions and optional expiration, and the dashboard uses a proxy so keys are not stored in browser localStorage.Read 8 Oct 2026 · github.com
Security guidance
The security policy recommends HTTPS, a reverse proxy, a long random API key salt, and regular database backups.Read 8 Oct 2026 · github.com
Storage
Prompt content is stored in Git, while metadata can use SQLite or PostgreSQL; filesystem, GitHub, and S3 storage drivers are listed.Read 8 Oct 2026 · github.com
Storage options
Prompt content can use filesystem, GitHub, or S3 storage, while metadata can use SQLite or PostgreSQL.Read 8 Oct 2026 · github.com

Strong points and limits

Strong points

  • Git-backed prompt versioning and rollback
  • Logs model, token, latency, and cost metadata
  • Dataset evaluations and prompt A/B tests
  • Scans for PII, keys, URLs, and IPs
  • Node.js and Python SDKs

Limits

  • Self-hosted deployment requires setup
  • Default rate limit is 300 requests per 60-second window
  • Security guidance recommends HTTPS and regular backups

Who it suits

PromptMetrics suits teams that want to host their own prompt registry and track LLM requests, agent activity, evaluations, or compliance risks. It is aimed at users comfortable choosing storage and deployment options and operating a self-hosted service.

Verdict

PromptMetrics combines prompt version control, LLM telemetry, evaluation tools, and compliance scanning in a self-hosted project. Its deployment and security guidance make operational setup part of adopting it.

Where Specifiction places it

Alternatives to PromptMetricsAll alternatives →