Scheduling jobs across Slurm clusters (and K8s, and cloud) from one place
Reddit r/MLOps1mo4 min read
A lot of ML teams end up with a mix: some Slurm clusters from the HPC side, a K8s cluster or two, maybe cloud GPUs for overflow. We wrote up how SkyPilot (open source) sits in front of all of them so a job is scheduled wherever there’s free capacity, using the same YAML regardless of backend. This post focuses on the multi-Slurm case but the same setup covers K8s. https://skypilot.ai/blog/multi-slurm Disclosure: I am the author. Happy to answer questions about how the scheduling and failover work submitted by /u/alex000kim [link] [comments]
