GPUStack vs HAMi

GPUStack is an open-source GPU cluster manager for deploying, governing, and scaling AI models across on-premise, cloud, or hybrid infrastructure. It provides OpenAI-compatible and Anthropic-compatible inference endpoints, with support for engines such as vLLM, SGLang, llama.cpp, TensorRT-LLM, and MindIE. Users can browse models from Hugging Face and ModelScope or supply local files, with automated compatibility checks. GPUStack can provision GPU instances with lifecycle controls, partitioning, SSH key injection, Jupyter access, and persistent S3 or NFS storage. Deployment options include on-premise, Kubernetes, and multi-cloud, including air-gapped deployments. The maker lists support for accelerators from NVIDIA, AMD, Ascend, T-head, Hygon, MetaX, Moore Threads, Cambricon, and Iluvatar. Security controls include RBAC, multi-tenancy, SSO, scoped API keys, IP allowlisting, token quotas, and usage analytics. GPUStack Community Edition is free. Worker nodes support Linux only; Windows users are directed to consider WSL2, and macOS worker nodes are not supported.

HAMi is open-source middleware for sharing and scheduling GPUs and other accelerators among AI workloads on Kubernetes. Workloads can request GPU memory and core limits through Kubernetes resource controls. Scheduling options include binpack, spread and numa-first, with binpack and spread available at node or GPU scope. The supported-device list spans NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren; isolation and partitioning capabilities vary by device. HAMi works with Kubernetes APIs, DRA and CDI, and documents scheduling integrations including Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler. Its node metrics endpoint exposes device memory and core utilization for Prometheus. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. NVIDIA limits use user-space library interception, which does not cover applications that bypass CUDA. Enforcement is described as soft and best-effort; the FAQ recommends MIG when hardware-enforced isolation is required.