HAMi vs Koordinator

HAMi is open-source middleware for sharing and scheduling GPUs and other accelerators among AI workloads on Kubernetes. Workloads can request GPU memory and core limits through Kubernetes resource controls. Scheduling options include binpack, spread and numa-first, with binpack and spread available at node or GPU scope. The supported-device list spans NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren; isolation and partitioning capabilities vary by device. HAMi works with Kubernetes APIs, DRA and CDI, and documents scheduling integrations including Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler. Its node metrics endpoint exposes device memory and core utilization for Prometheus. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. NVIDIA limits use user-space library interception, which does not cover applications that bypass CUDA. Enforcement is described as soft and best-effort; the FAQ recommends MIG when hardware-enforced isolation is required.

Koordinator is a free, self-hosted QoS scheduling system for Kubernetes microservices, AI, big-data, audio, and video workloads. It adds priority and quality-of-service controls so different pods and workload types can share nodes. Application profiling supports resource overcommitment while preserving QoS guarantees, and fine-grained isolation targets latency-sensitive work alongside batch jobs. Customizable policies cover Web Service, Spark, Presto, TensorFlow, and PyTorch. QoSManager tunes pod isolation using profiling, interference detection, and SLO settings. Koord-Queue supports TFJob, PyTorchJob, Spark, Argo Workflow, Ray, and native Kubernetes Jobs, with Kueue AdmissionCheck compatibility. YARN Copilot connects Hadoop YARN jobs with Kubernetes pods. Koordinator works alongside kube-scheduler, using control-plane and node components without invasive native changes. Helm 3.5 or newer and Kubernetes 1.18 or newer are required.