HTCondor vs SkyPilot

HTCondor is a free high-throughput computing system that coordinates networked machines to run submitted jobs. It finds available machines, matches their resource offers to job requirements and preferences through ClassAds, and can restart a job elsewhere if its current machine becomes unavailable. It is suited to submitting large batches, including thousands of runs across different data sets, and can use otherwise idle machines in a pool. Users do not need login accounts on machines running their jobs; file transfers and split execution support that model. HTCondor can draw on clusters, cloud resources, and international grids, and flocking can connect multiple installations. It supports transfers to and from S3-compatible storage, including Google Cloud Storage through its S3 interoperability API, but not entire buckets or directories. The software is licensed under Apache License 2.0, includes Python bindings, and offers user and administrator guides. Community support is available through a mailing list; paid contract support offers guaranteed resolution and faster turnaround. Security fixes cover current feature and LTS versions, with the previous LTS supported for approximately twelve months after a new LTS release.

SkyPilot is an AI compute platform that manages Kubernetes, Slurm, virtual machines, on-premises systems and cloud resources through one interface. Listed workloads include development, pre-training, post-training and reinforcement learning, batch inference, sandboxes and inference endpoints. Scheduling provides priority queues, GPU sharing, quotas and intelligent placement across workloads. Fleet health checks can run proactively or reactively, with automatic remediation for GPU and NCCL faults. Integrations include AWS, GCP, Azure, Nebius, CoreWeave, Lambda, RunPod, Kubernetes, Slurm and on-premises infrastructure. SkyPilot can run standalone or connect to a remote API server for multi-user collaboration. Enterprise controls include SSO onboarding, RBAC, workspaces, policy enforcement, a unified dashboard, idle-compute stopping and cost reporting. Deployment options include BYOC, BYOK, private VPC and air-gapped environments. The project began as open source at UC Berkeley's Sky Computing Lab and is offered as open source. Documentation says Slurm support remains under active development and SkyServe on Slurm is not supported.