SkyPilot and Hugging Face Launch Zero-Egress Storage Integration for Multi-Cloud GPU Workloads

SkyPilot and Hugging Face have jointly shipped a storage integration that lets GPU jobs running on any cloud directly mount models and datasets from the Hugging Face Hub via a single `hf://` URL — with no egress fees and no need to maintain per-cloud copies of data. Teams can now point a SkyPilot task at a Hugging Face Bucket or any Hub repository (model, dataset, or Space) using their existing `HF_TOKEN`, and SkyPilot will schedule the compute across more than 20 clouds, Kubernetes, Slurm, or on-premises clusters, wherever capacity is available.

The integration uses Hugging Face's `hf-mount` FUSE backend — with fixes upstreamed by the Hugging Face team to support unprivileged containers — so buckets and repos appear as local paths alongside SkyPilot's other cloud storage mounts. Reads are lazy and byte-range aware, meaning a process can begin work before a full file downloads, keeping GPUs active. Buckets are backed by Xet, which deduplicates storage at the chunk level so incremental checkpoints and model variants only store and transfer what has changed.