Apache YuniKorn vs TORQUE Resource Manager

Apache YuniKorn is a resource scheduler for batch, data, and machine learning workloads on Kubernetes. It is a Kubernetes-compatible alternative to the default scheduler and is described as transparent to existing Kubernetes applications. Hierarchical queues provide fine-grained tenant controls that can reflect an organization’s structure. Scheduling features include gang scheduling, FIFO and Fair job ordering, resource fairness, reservations, and preemption. A web UI centralizes cluster capacity, utilization, and application information. YuniKorn exposes metrics through Prometheus, and project guidance covers monitoring with Prometheus and Grafana. The project recommends deployment to an existing Kubernetes cluster using its Helm chart and Helm 3 or later. The software is free under the Apache License 2.0. Releases include source tarballs, which must be built before use, and prebuilt Docker images. The project advises checking release artifacts for tampering with GPG or SHA-512. Help is available through the YuniKorn Slack #yunikorn-user channel or the user mailing list.

TORQUE Resource Manager is an open-source resource and queue manager for compute clusters, based on the original PBS resource manager. The project is free, with Linux, macOS, API and self-hosted options. TORQUE 4.0 release notes state that it can support clusters with more than 15,000 compute nodes; that release also introduced a multithreaded pbs_server intended to improve command response and job throughput. An optional MOM hierarchy can reduce update traffic from compute nodes to pbs_server and lets administrators manage network communication. In version 4.0, component communication moved to TCP/IP, removing UDP and RPP. The trqauthd daemon authorizes client connections to pbs_server and must run as root on client hosts. Linux builds can optionally use control groups and cpusets, and documentation describes integration with Moab Workload Manager. The project is built from source using configure and make; installation instructions also cover RPM and self-extracting package creation. Its license terms are in the repository's PBS_License.txt. A compatibility caveat for version 4.0 is that its components cannot communicate with earlier TORQUE versions, so rolling upgrades will not work.