KAI Scheduler vs Kueue

KAI Scheduler is a self-hosted, open-source Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, and inference. It supports interactive notebooks through multi-node distributed training in one cluster and is designed for large GPU clusters and high-throughput workloads. Hierarchical queue trees provide quotas, limits, priorities, and borrowing between teams. Gang scheduling places distributed workloads together when resources are available, while elastic workloads change replica counts within configured minimum and maximum values. GPU sharing includes time slicing, MPS, and MIG. Other scheduling controls include topology-aware placement, time-based fairshare, and priority-based preemption. KAI documents integrations with Ray on Kubernetes and Grove. It runs on cloud or on-premises infrastructure, including Karpenter environments. Installation uses a production Helm chart, source builds, or ArgoCD, with a running Kubernetes cluster, Helm CLI, and NVIDIA GPU Operator required for GPU requests. The Apache-2.0 project is also a CNCF sandbox project.

Kueue is a free Kubernetes project that manages quotas and job resource consumption, deciding when workloads wait, start, or are preempted. It installs on a standard Kubernetes cluster without replacing existing components. Its scheduling tools include priority-based queues, fair sharing, quota borrowing between teams, resource flavor fungibility, preemption policies, and all-or-nothing scheduling. Built-in integrations cover Kubernetes Jobs and Pods, Kubeflow training jobs, Ray jobs and clusters, JobSet, AppWrapper, Deployments, StatefulSets, and LeaderWorkerSet. MultiKueue can find capacity and dispatch workloads across clusters, while topology-aware scheduling accounts for data center layout to improve communication throughput between pods. Kueue provides Prometheus metrics and an endpoint for viewing pending workloads. Installation instructions cover kubectl, Helm, and source builds. The guide recommends Kubernetes 1.34 or newer and officially supports and tests only those versions. Kueue also provides administrator and user ClusterRoles, with resource-specific roles that can be composed into custom roles. Its webhook server includes internal certificate management, and the installation guide links to cert-manager instructions.