dstack vs HAMi

dstack is an open-source orchestration layer for AI workloads across GPU clouds, Kubernetes, virtual machines, and bare-metal clusters. YAML configurations define fleets, development environments, tasks, services, presets, and volumes, while dstack handles provisioning and job scheduling, including auto-scaling, port forwarding, and ingress. Supported accelerators include NVIDIA, AMD, TPU, and Tenstorrent. Documented backends include AWS, Azure, GCP, Kubernetes, GPU cloud providers, remote SSH hosts, and experimental Slurm. Tasks can use frameworks such as accelerate, torchrun, Ray, and Spark. Users manage resources through the CLI or HTTP API, and the server can run on a laptop or another environment that can access the clusters in use. Services can publish inference endpoints through gateways with HTTPS, custom domains, auto-scaling, and rate limits. The self-hosted open-source stack is free. Server data and project secrets are stored in plaintext by default unless administrators configure AES-256-GCM encryption. TPU support is limited to single-host instances with up to eight cores. GPU Marketplace usage is prepaid, with prices that vary by GPU and provider and appear in the console before provisioning.

HAMi is open-source middleware for sharing and scheduling GPUs and other accelerators among AI workloads on Kubernetes. Workloads can request GPU memory and core limits through Kubernetes resource controls. Scheduling options include binpack, spread and numa-first, with binpack and spread available at node or GPU scope. The supported-device list spans NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren; isolation and partitioning capabilities vary by device. HAMi works with Kubernetes APIs, DRA and CDI, and documents scheduling integrations including Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler. Its node metrics endpoint exposes device memory and core utilization for Prometheus. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. NVIDIA limits use user-space library interception, which does not cover applications that bypass CUDA. Enforcement is described as soft and best-effort; the FAQ recommends MIG when hardware-enforced isolation is required.