dstack Free plan 8.9 Visit KAI Scheduler
kai-scheduler.dev
What it is
KAI Scheduler is a self-hosted, open-source Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, and inference. It supports interactive notebooks through multi-node distributed training in one cluster and is designed for large GPU clusters and high-throughput workloads. Hierarchical queue trees provide quotas, limits, priorities, and borrowing between teams. Gang scheduling places distributed workloads together when resources are available, while elastic workloads change replica counts within configured minimum and maximum values. GPU sharing includes time slicing, MPS, and MIG. Other scheduling controls include topology-aware placement, time-based fairshare, and priority-based preemption. KAI documents integrations with Ray on Kubernetes and Grove. It runs on cloud or on-premises infrastructure, including Karpenter environments. Installation uses a production Helm chart, source builds, or ArgoCD, with a running Kubernetes cluster, Helm CLI, and NVIDIA GPU Operator required for GPU requests. The Apache-2.0 project is also a CNCF sandbox project.
Spec sheet
- Purpose
- KAI Scheduler is a Kubernetes-native scheduler for AI workloads that optimizes GPU allocation across data processing, training, and inference while keeping resource shares fair across teams.Read 7 Oct 2026 · kai-scheduler.dev
- Deployment
- The project provides production Helm installation, source build, and ArgoCD GitOps installation options for Kubernetes.Read 7 Oct 2026 · github.com
- Integrations
- The project states it is natively integrated with KubeRay for Ray workloads on Kubernetes and describes integrations with Grove and Dynamo for complex multi-component workloads.Read 7 Oct 2026 · github.com
- Security
- Security vulnerabilities should be reported through GitHub Security Advisories rather than public GitHub issues.Read 5 Oct 2026 · github.com
- Support
- The project directs users to GitHub issues for bugs, feature requests, or technical help, and also lists CNCF Slack, community calls, and a mailing list.Read 7 Oct 2026 · github.com
- License
- The public GitHub repository identifies the project as open source and lists an Apache-2.0 license.Read 7 Oct 2026 · github.com
- Cloud and on-premises
- The project describes compatibility with dynamic cloud infrastructure, including Karpenter autoscalers, and static on-premises deployments.Read 7 Oct 2026 · github.com
- Elastic workloads
- Elastic workloads can grow or shrink within configured minimum and maximum replica counts as capacity changes.Read 5 Oct 2026 · kai-scheduler.dev
- Fairness
- Time-based fairshare tracks historical GPU usage over a configurable window to distribute over-quota resources fairly over time.Read 7 Oct 2026 · kai-scheduler.dev
- Gang and elastic scheduling
- Gang scheduling places distributed workloads all at once, while elastic workloads can grow or shrink within minimum and maximum replica counts as capacity allows.Read 7 Oct 2026 · kai-scheduler.dev
- Gang scheduling
- Gang scheduling places all pods in a distributed workload together or waits until resources are available.Read 5 Oct 2026 · github.com
- GPU sharing
- GPU sharing supports time slicing, MPS, and MIG so workloads can share GPU devices.Read 7 Oct 2026 · kai-scheduler.dev
- GPU sharing limitation
- The GPU-sharing documentation says the default NonMemoryEnforced mode schedules fractional GPU workloads without KAI enforcing runtime GPU memory isolation.Read 7 Oct 2026 · github.com
- Hierarchical queues
- Multi-level queue trees support quotas, limits, and over-quota borrowing across teams.Read 7 Oct 2026 · kai-scheduler.dev
- Intended workloads
- It supports workloads ranging from interactive notebooks to multi-node distributed training in the same cluster.Read 7 Oct 2026 · kai-scheduler.dev
- License and project
- KAI Scheduler is an open-source Apache-2.0 project and a CNCF sandbox project.Read 5 Oct 2026 · github.com
- Prerequisite
- The installation guide says NVIDIA GPU-Operator is required to schedule workloads requesting GPU resources.Read 5 Oct 2026 · github.com
- Prerequisites
- The installation guide lists a running Kubernetes cluster and Helm CLI, and requires NVIDIA GPU Operator for workloads requesting GPU resources.Read 7 Oct 2026 · github.com
- Priority and preemption
- Per-queue and per-workload priorities let critical jobs preempt while non-critical jobs back off.Read 7 Oct 2026 · kai-scheduler.dev
- Queues
- Hierarchical queues support quotas, limits, priorities, and borrowing resources across teams.Read 5 Oct 2026 · kai-scheduler.dev
- Scale
- The project says it is designed and continuously tested for large GPU clusters with thousands of nodes and high-throughput workloads.Read 5 Oct 2026 · kai-scheduler.dev
- Support channel
- The project directs users to open GitHub issues for bugs, feature suggestions, or technical help.Read 5 Oct 2026 · github.com
- Supported infrastructure
- The project states it supports cloud and on-premise deployments, including cloud auto-scalers such as Karpenter.Read 5 Oct 2026 · github.com
- Topology-aware placement
- KAI supports topology-aware scheduling, including hierarchical topology-aware scheduling for hierarchical PodGroups.Read 7 Oct 2026 · kai-scheduler.dev
Strong points and limits
Strong points
- Hierarchical queues provide quotas, priorities, and borrowing
- Gang scheduling coordinates distributed workload pods
- GPU sharing supports time slicing, MPS, and MIG
- Cloud and on-premises deployments are supported
- Apache-2.0 open-source project
Limits
- Requires a running Kubernetes cluster and Helm
- GPU requests require NVIDIA GPU Operator
- Default sharing mode does not enforce runtime GPU memory isolation
- Self-hosted deployment requires infrastructure management
Who it suits
KAI Scheduler suits teams running GPU workloads on Kubernetes across shared cloud or on-premises clusters. It is aimed at organizations managing distributed, elastic, or multi-team AI jobs.
Verdict
KAI Scheduler combines queue controls, coordinated placement, elastic replicas, and GPU sharing for Kubernetes AI workloads. Its prerequisites and self-hosted model make it suited to teams already operating Kubernetes and NVIDIA infrastructure.
Where Specifiction places it
- GPU Cluster Management Software#13 of 217.4
