dstack

Overview

dstack is an open-source orchestration layer for teams running AI workloads across cloud and on-premises compute. It is best suited to administrators who need one workflow for provisioning and scheduling across different accelerators and environments. Its breadth is useful, but plaintext storage defaults and the single-host TPU ceiling deserve attention.

The server can run wherever it can reach the cloud and on-premises clusters in use, including a laptop. Users define fleets, development environments, tasks, services, presets and volumes in YAML, then manage resources with the CLI or HTTP API. That makes dstack infrastructure software, not a general-purpose desktop app; its listed platforms are API, Linux, macOS, self-hosted, web and Windows.

Key features

Provisioning and scheduling

dstack combines infrastructure provisioning with job scheduling across GPU clouds, Kubernetes, virtual machines and bare-metal clusters. Auto-scaling, port forwarding and ingress support workloads ranging from tasks to services. Kubernetes support and GPU utilization metrics will matter to teams operating mixed environments, but the absence of quota controls is a drawback where administrators need built-in resource caps.

Accelerators and frameworks

NVIDIA, AMD, TPU and Tenstorrent accelerators are supported out of the box. The maker describes compatibility with any hardware, open-source tools and frameworks; the task guide specifically names accelerate, torchrun, Ray and Spark for distributed work. This breadth suits teams with varied stacks better than a workflow tied to one accelerator family. TPU support is narrower: dstack currently handles single-host TPUs only, up to eight cores per instance.

Services, interfaces and security

Services can expose model inference through endpoints, with gateways providing HTTPS, custom domains, auto-scaling and rate limits. The CLI covers resource management, while the HTTP API enables functions beyond the CLI and direct server integrations. The server can connect to AWS, Azure, GCP, Kubernetes, multiple GPU cloud providers and remote SSH hosts; Slurm support is experimental.

Security needs deliberate configuration. Server data and project-scoped secrets are stored in plaintext by default; administrators can configure AES-256-GCM encryption for stored data. Secrets are managed by project admins, so teams should consider the storage default before placing sensitive credentials in a deployment.

Pricing

PlanPriceWhat it includesdstack OSSfreeOpen-source orchestration stack; self-hosted.dstack Sky GPU MarketplaceCustom pricingOn-demand and spot GPU compute, billed pay-as-you-go per GPU-hour, with prepaid credits.The self-hosted OSS plan suits teams that want the orchestration stack without a plan fee and can operate it themselves. dstack Sky's BYOC mode currently carries no charge; Marketplace compute instead draws on prepaid credits, and rates vary by GPU and provider, with prices shown in the console before provisioning. That model avoids a fixed usage price but requires checking the resource rate before launching work. dstack Factory is the commercial extension, adding advanced multi-tenancy, usage metering, billing automation and optimized inference presets for frontier open models; it uses custom pricing.

Platforms

dstack lists API, Linux, macOS, self-hosted, web and Windows. Its hybrid deployment model is useful when the control server needs to reach both cloud resources and on-premises clusters. Support for cloud GPUs and Kubernetes broadens deployment options, while GPU utilization metrics help monitor use; without quota controls, teams needing enforced caps must account for that operational gap.

Who it's for

dstack is a strong fit for AI infrastructure teams coordinating workloads across heterogeneous accelerators, cloud GPU providers and on-premises systems, especially when YAML-defined workloads and a CLI or API suit their operations. It is less suited to teams requiring built-in quota enforcement, encrypted storage without administrator setup, or multi-host TPU workloads.

Pros and cons

Pro: One orchestration layer spans GPU clouds, Kubernetes, VMs and bare metal, reducing the need to split provisioning and scheduling across environment-specific workflows.

Pro: Out-of-the-box support for four accelerator families and named distributed frameworks accommodates varied AI infrastructure.

Pro: Service gateways include HTTPS, auto-scaling, custom domains and rate limits for published inference endpoints.

Con: Server data and secrets default to plaintext, so administrators must configure encryption for stored data.

Con: TPU workloads are limited to a single host and eight cores per instance.

Con: Quota controls are absent, limiting built-in control over resource allocation.

Alternatives

Backend.AI is worth considering for users seeking a freemium option with a free trial and a home-user open-source plan.

ClearML may suit teams that want a freemium product with a self-hosted version described by its maker as fully open source.

GPUStack is another free, open-source GPU cluster manager for teams seeking that narrower focus.

HTCondor is a free alternative for readers comparing cluster-management options.

NVIDIA ShadowPlay is a free Windows option with a supported GeForce GPU requirement.

HAMi is a free Linux and self-hosted choice for GPU virtualization middleware on Kubernetes.

Kueue is a free open-source Kubernetes project for readers focused on that platform.

OpenPBS offers a free AGPL 3.0 open-source edition, with community forum support carrying no guarantees.

GPU Cluster Management Software is a broader category comparison.

Verdict

Choose dstack if you need open-source orchestration for AI workloads across mixed cloud and on-premises infrastructure, and value its broad accelerator support and combined provisioning and scheduling. Look elsewhere if built-in quotas, encrypted-by-default storage or multi-host TPU support are requirements.