HAMi vs TORQUE Resource Manager

HAMi is open-source middleware for sharing and scheduling GPUs and other accelerators among AI workloads on Kubernetes. Workloads can request GPU memory and core limits through Kubernetes resource controls. Scheduling options include binpack, spread and numa-first, with binpack and spread available at node or GPU scope. The supported-device list spans NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren; isolation and partitioning capabilities vary by device. HAMi works with Kubernetes APIs, DRA and CDI, and documents scheduling integrations including Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler. Its node metrics endpoint exposes device memory and core utilization for Prometheus. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. NVIDIA limits use user-space library interception, which does not cover applications that bypass CUDA. Enforcement is described as soft and best-effort; the FAQ recommends MIG when hardware-enforced isolation is required.

TORQUE Resource Manager is an open-source resource and queue manager for compute clusters, based on the original PBS resource manager. The project is free, with Linux, macOS, API and self-hosted options. TORQUE 4.0 release notes state that it can support clusters with more than 15,000 compute nodes; that release also introduced a multithreaded pbs_server intended to improve command response and job throughput. An optional MOM hierarchy can reduce update traffic from compute nodes to pbs_server and lets administrators manage network communication. In version 4.0, component communication moved to TCP/IP, removing UDP and RPP. The trqauthd daemon authorizes client connections to pbs_server and must run as root on client hosts. Linux builds can optionally use control groups and cpusets, and documentation describes integration with Moab Workload Manager. The project is built from source using configure and make; installation instructions also cover RPM and self-extracting package creation. Its license terms are in the repository's PBS_License.txt. A compatibility caveat for version 4.0 is that its components cannot communicate with earlier TORQUE versions, so rolling upgrades will not work.