HAMi vs KAI Scheduler
HAMi
project-hami.ioKAI Scheduler
kai-scheduler.devSide by side
| Figure | HAMi | KAI Scheduler |
|---|---|---|
| Price from | Free | — |
| Free plan | Yes | — |
| Free trial | — | — |
| Open source | — | — |
| Linux | Yes | — |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 0 |
| Spec score · GPU Cluster Management Software | 8.3 (#5) | 7.4 (#13) |
| Deployment | The quick start installs HAMi with Helm and requires Kubernetes, Helm, kubectl, and installation permissions. | The project provides production Helm installation, source build, and ArgoCD GitOps installation options for Kubernetes. |
| Integrations | The site lists Kubernetes, Volcano, Kueue, Koordinator and KAI Scheduler in its Kubernetes scheduling ecosystem. | The project states it is natively integrated with KubeRay for Ray workloads on Kubernetes and describes integrations with Grove and Dynamo for complex multi-component workloads. |
| Purpose | HAMi is open-source, cloud-native GPU virtualization middleware for sharing, isolating, and scheduling heterogeneous accelerators on Kubernetes. | KAI Scheduler is a Kubernetes-native scheduler for AI workloads that optimizes GPU allocation across data processing, training, and inference while keeping resource shares fair across teams. |
| Support | The site links to documentation, tutorials, Discord and the `#hami-dev` Slack channel for community resources. | The project directs users to GitHub issues for bugs, feature requests, or technical help, and also lists CNCF Slack, community calls, and a mailing list. |
Plans
In their own words
HAMi
HAMi is open-source middleware for sharing and scheduling GPUs and other accelerators among AI workloads on Kubernetes. Workloads can request GPU memory and core limits through Kubernetes resource controls. Scheduling options include binpack, spread and numa-first, with binpack and spread available at node or GPU scope. The supported-device list spans NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren; isolation and partitioning capabilities vary by device. HAMi works with Kubernetes APIs, DRA and CDI, and documents scheduling integrations including Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler. Its node metrics endpoint exposes device memory and core utilization for Prometheus. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. NVIDIA limits use user-space library interception, which does not cover applications that bypass CUDA. Enforcement is described as soft and best-effort; the FAQ recommends MIG when hardware-enforced isolation is required.
KAI Scheduler
KAI Scheduler is a self-hosted, open-source Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, and inference. It supports interactive notebooks through multi-node distributed training in one cluster and is designed for large GPU clusters and high-throughput workloads. Hierarchical queue trees provide quotas, limits, priorities, and borrowing between teams. Gang scheduling places distributed workloads together when resources are available, while elastic workloads change replica counts within configured minimum and maximum values. GPU sharing includes time slicing, MPS, and MIG. Other scheduling controls include topology-aware placement, time-based fairshare, and priority-based preemption. KAI documents integrations with Ray on Kubernetes and Grove. It runs on cloud or on-premises infrastructure, including Karpenter environments. Installation uses a production Helm chart, source builds, or ArgoCD, with a running Kubernetes cluster, Helm CLI, and NVIDIA GPU Operator required for GPU requests. The Apache-2.0 project is also a CNCF sandbox project.