Skip to content
SPECIFICTION
Get listed

Experimently

getexperimently.com

Free plan Open source WebWindowsmacOSLinuxiOSAndroid
Visit getexperimently.com
Spec score 8.7 #9 of 52 in A/B Testing Tools

What it is

Experimently is an open-source experimentation platform for A/B testing and feature flags that runs on infrastructure you manage. Its Free Preview costs nothing when self-hosted on your AWS account, with all features included; there is no hosted tier. The platform supports A/B and multivariate tests, sequential testing, CUPED, multi-armed bandits, Bayesian testing, split URL testing and LLM/model evaluation. Rollout controls include sticky assignment, staged rollouts, mutual-exclusion groups, a global holdout and automatic rollback based on error-rate and latency thresholds. Targeting offers more than 20 operators, while analysis includes frequentist and Bayesian methods, sequential mSPRT and segment breakdowns with Bonferroni correction. SDKs are listed for many programming languages and mobile platforms, but the maker says they are not yet on package registries; installation directions are in each guide. Deployment options include Docker Compose, Kubernetes through Helm, or an AWS CDK app for services in your AWS account. Assignment data stays on your infrastructure. Integrations include data platforms, business tools and email. The platform lists security and compliance features including role-based access control, SSO and audit logging.

Spec sheet

What it does
Experimently is an open-source experimentation and feature-flag platform deployed on infrastructure controlled by the customer.Read 14 Oct 2026 · getexperimently.com
Product
Experimently is an open-source experimentation platform for A/B testing and feature flags that runs on your own infrastructure.Read 5 Oct 2026 · getexperimently.com
Deployment
Users can deploy with Docker Compose, Kubernetes using a Helm chart, or an AWS CDK app targeting ECS Fargate, Aurora PostgreSQL, and ElastiCache in their own AWS account.Read 5 Oct 2026 · getexperimently.com
Integrations
The documentation lists connections for Snowflake, BigQuery, Redshift, Databricks, ClickHouse, MySQL, Jira, Salesforce, GitHub, Slack, and email.Read 5 Oct 2026 · getexperimently.com
Support
The maker lists [email protected] as its contact email.Read 5 Oct 2026 · getexperimently.com
A/B and multivariate testing
The product documentation covers A/B, multivariate, sequential, split-URL, Bayesian and bandit experiments.Read 14 Oct 2026 · getexperimently.com
Compliance reporting
The documentation provides export functions for SOC 2 and ISO 27001 reports and describes HIPAA-related encryption, retention and BAA features.Read 14 Oct 2026 · getexperimently.com
CUPED
CUPED variance reduction is documented as a method for reducing variance using pre-experiment covariates.Read 14 Oct 2026 · getexperimently.com
Data handling
The maker says assignment data stays on the user's own infrastructure and that there is no hosted tier.Read 5 Oct 2026 · getexperimently.com
Data integrations
Documentation lists AWS, Snowflake, BigQuery, Redshift, Databricks, ClickHouse and MySQL connectors.Read 14 Oct 2026 · getexperimently.com
Deployment options
Customers can run published containers with Docker Compose, deploy to Kubernetes with Helm, or use an AWS CDK deployment.Read 14 Oct 2026 · getexperimently.com
Documentation depth
The documentation site includes quick starts, API schemas, SDK guides, integration references, security topics, deployment instructions and statistics references.Read 14 Oct 2026 · getexperimently.com
Experiment controls
The platform supports sticky assignment, staged rollouts, mutual-exclusion groups, global holdouts and automatic rollback triggered by error-rate or latency thresholds.Read 14 Oct 2026 · getexperimently.com
Experiments and rollouts
It supports sticky assignment, staged rollouts, mutual-exclusion groups, a global holdout, and automatic rollback based on error-rate and latency thresholds.Read 5 Oct 2026 · getexperimently.com
Independent maintainer
PyPI identifies amarkanday as the sole maintainer of the placeholder experimently package.Read 14 Oct 2026 · pypi.org
Independent package maturity
PyPI classifies the package as development status 1, Planning, and lists Python 3.9 or newer as a requirement.Read 14 Oct 2026 · pypi.org
Independent performance target
The project article sets a target of under 50 ms at p99 for feature-flag evaluation and more than 10,000 assignments per second.Read 14 Oct 2026 · medium.com
Independent planned architecture
The project article describes an AWS architecture using ECS/Fargate, Lambda, Aurora PostgreSQL, DynamoDB, ElastiCache, S3, Kinesis, EventBridge, OpenSearch, Glue and Athena.Read 14 Oct 2026 · medium.com
Independent project date
The product-management article describing the platform was published March 28, 2025.Read 14 Oct 2026 · medium.com
Independent project origin
Ashish Markanday wrote that building the platform was a personal project intended to create a production-quality experimentation system with AI assistance in roughly three to four months.Read 14 Oct 2026 · medium.com
Independent risk: complexity
The project article identifies high-throughput real-time evaluation, data integrity, integration complexity and statistical validity as material implementation risks.Read 14 Oct 2026 · medium.com
Integration API
Integration endpoints support Jira, Salesforce and GitHub, including webhook receivers and role-based permissions.Read 14 Oct 2026 · getexperimently.com
JavaScript SDK
The JavaScript client supports browser and Node.js use, variant assignment, feature-flag evaluation and event tracking.Read 14 Oct 2026 · getexperimently.com
Licence
The core platform is Apache-2.0 licensed; the SDKs are MIT licensed.Read 14 Oct 2026 · getexperimently.com
License and price
The platform is Apache-2.0 licensed, and the maker says its build profiles have no price; its Free Preview is listed at $0/month.Read 5 Oct 2026 · getexperimently.com
Limit: hosting responsibility
There is no hosted tier, so users must provision and operate the deployment in their own infrastructure or AWS account.Read 14 Oct 2026 · getexperimently.com
Limit: SDK registry availability
The homepage says JavaScript and Python SDKs are beta releases while the other listed SDKs are not yet published in package registries and must be installed from the repository.Read 14 Oct 2026 · getexperimently.com
LLM evaluation
Documentation describes comparing prompts, model variants and agent configurations against business metrics.Read 14 Oct 2026 · getexperimently.com
Maker identity
The homepage credits Ashish Markanday as the builder and provides [email protected] as the contact address.Read 14 Oct 2026 · getexperimently.com
Public API
The REST API uses OAuth2 bearer tokens and API keys and documents experiment, feature-flag, tracking, audit and integration endpoints.Read 14 Oct 2026 · getexperimently.com
Python SDK status
The PyPI project page says the published 0.0.0 package is only a placeholder and instructs users to install the SDK from the GitHub source until a real release.Read 14 Oct 2026 · pypi.org
Quick-start requirements
The quick-start guide requires Python 3.9 or newer, Node.js 18 or newer, Docker Compose 24 or newer and Git.Read 14 Oct 2026 · getexperimently.com
Real-time results
The documentation lists live experiment-result streaming over WebSockets with continuously updated p-values, lift and significance.Read 14 Oct 2026 · getexperimently.com
SDK availability
The maker says its SDKs are not yet on npm, PyPI, or other package registries and that installation instructions are in each SDK guide.Read 5 Oct 2026 · getexperimently.com
SDK coverage
Official SDK guides cover JavaScript, Python, Java, React, Go, iOS, Android, Flutter, React Native, Edge, Ruby, PHP, .NET and Elixir.Read 14 Oct 2026 · getexperimently.com
SDKs
SDKs are listed for JavaScript, React, React Native, Edge, Python, Go, Java, Android, iOS, Flutter, Ruby, PHP, .NET, and Elixir, plus OpenFeature providers for JavaScript and Python.Read 5 Oct 2026 · getexperimently.com
Security and compliance
The documentation lists HMAC-SHA256 audit logging, role-based access control, scoped API keys, SAML and OIDC/OAuth2 SSO, and SOC 2, ISO 27001, and HIPAA compliance features.Read 5 Oct 2026 · getexperimently.com
Security controls
The documentation lists signed audit events, HMAC-SHA256 audit signatures, scoped API keys, key rotation, revocation and role-based access control.Read 14 Oct 2026 · getexperimently.com
Self-hosting components
The local deployment consists of Postgres, Redis, an API service and a dashboard, with database creation and seeding during startup.Read 14 Oct 2026 · getexperimently.com
SSO
Supported identity options listed in the documentation include SAML 2.0 and OIDC/OAuth2 for Okta, Azure AD, Google Workspace, GitHub and OneLogin.Read 14 Oct 2026 · getexperimently.com
Statistical analysis
Experimently provides frequentist and Bayesian analysis, sequential mSPRT testing, multi-armed bandits and per-segment results with Bonferroni correction.Read 14 Oct 2026 · getexperimently.com
Statistics
Analysis includes frequentist and Bayesian methods, sequential mSPRT testing, multi-armed bandits, and segment breakdowns with Bonferroni correction.Read 5 Oct 2026 · getexperimently.com
Targeting
Its targeting engine advertises more than 20 operators, including semantic-version comparison, geo-distance, time windows, JSON-path and array operations.Read 14 Oct 2026 · getexperimently.com

Plans and prices

Free Preview Free Self-hosted on your AWS account · all features included Read 5 Oct 2026 · getexperimently.com
Free Preview Free $0/month Self-hosted on your AWS account · all features included · unlimited A/B experiments Read 14 Oct 2026 · getexperimently.com

Every Experimently plan and price →

Strong points and limits

Strong points

  • Free Preview costs nothing.
  • All features are included in the self-hosted preview.
  • Supports multiple statistical testing approaches.
  • Assignment data stays on your infrastructure.
  • Offers several deployment options.

Limits

  • There is no hosted tier.
  • SDKs are not yet on package registries.
  • Users must deploy and manage their own infrastructure.

Who it suits

It suits teams that want to run experiments and feature flags on infrastructure they manage. It also fits teams able to deploy it through Docker Compose, Kubernetes or AWS CDK.

Verdict

Experimently provides a broad set of testing, targeting and rollout controls for self-managed deployments. Its free Free Preview includes all features, but setup depends on infrastructure you operate.

Where Specifiction places it

Alternatives to ExperimentlyAll alternatives →