Scale-to-zero is hard to offer for large models. When a model takes minutes to cold-start, operators either keep a GPU allocated while it sits idle or make customers wait while it wakes up.
Saturn Cloud, an NVIDIA DSX ecosystem partner, helps AI cloud operators turn GPU infrastructure into branded inference services. Faster model startup could help operators offer a broader model catalog while reducing the need to reserve idle GPUs for models waiting for requests.
We benchmarked NVIDIA Snapshot on an already-running Nebius NVIDIA B300 node. Snapshot is an open-source, modular component of the NVIDIA DSX AI Factory Platform that accelerates startup and recovery by restoring initialized GPU workloads. Qwen3-14B, Qwen3-32B, and GPT-OSS-120B each used one GPU; GLM 5.3 used all eight. Starting from downloaded weights took 1m51s to 8m29s to answer a short request. Workers restored from previously captured Snapshots answered in 6s to 34s. These are initial results from one paired run per Qwen or GPT-OSS model and two paired GLM runs, not averages.
What NVIDIA Snapshot does
Snapshot enables a checkpoint-and-restore approach for inference workloads on Kubernetes. It uses CRIU (Checkpoint/Restore in Userspace) and NVIDIA’s cuda-checkpoint utility to capture a worker’s fully initialized state, both the CPU-side process state and the GPU-side device memory, after it has cold-started once. Later workers restore from that checkpoint instead of cold-starting again.
The first start still loads the weights, warms the kernels, and captures the execution graphs. After a checkpoint is created, a compatible worker can resume that initialized state.
What we tested
We tested Qwen3-14B, Qwen3-32B, GPT-OSS-120B, and RadixArk/GLM-5.3-NVFP4 in a Nebius us-north1 Kubernetes cluster. The node had eight NVIDIA B300 SXM6 PC GPUs. The Qwen and GPT-OSS workers each used one GPU; GLM 5.3 used all eight with tensor and expert parallelism. The models ran serially. Six 3.84 TB local NVMe SSDs were striped into a RAID0 volume that held both downloaded model weights and Snapshot artifacts. The node used NVIDIA DRIVE 615.71.09. All workers used vLLM 0.31.0 and Snapshot 0.2.0-rc.3. GPT-OSS-120B used FlashInfer CUTLASS with MXFP4 weights and MXFP8 activations.
Results
The table compares time from serving-worker Pod creation to the first successful response to the same short request through a small in-pod /generate API backed by vLLM. The cold and restored workers for each model ran on the same already-running B300 node, using one GPU for Qwen and GPT-OSS or eight GPUs for GLM 5.3. Their container image was cached, and their model weights were already downloaded to the local SSD. The startup still reads the weights, loads them into GPU memory, and initializes vLLM. We ran an initial cold start, captured the Snapshot, tested restore, and then repeated the cold start with the node’s file cache warm. The table uses that warm-cache cold run. The startup comparison excludes GPU-node provisioning, image pulls, model downloads, and Snapshot creation.
The final column reports one-time Snapshot capture from the same B300 experiment. Capture is not included in either startup time or the speedup. The Snapshots occupied about 34 GiB, 68 GiB, 73 GiB, and 564 GiB on local SSD for Qwen3-14B, Qwen3-32B, GPT-OSS-120B, and GLM 5.3, respectively.
| Model | Cold to reply | Restore to reply | Speedup | Snapshot creation |
|---|---|---|---|---|
| Qwen3-14B BF16 | 1m51s | 6s | 18.5× | 4s |
| Qwen3-32B BF16 | 2m11s | 8s | 16.4× | 5s |
| GPT-OSS-120B | 2m44s | 9s | 18.2× | 6s |
| GLM 5.3 NVFP4 | 8m22–8m29s | 33–34s | 15.0–15.2× | 41s |
Why this matters for operators
An eight-GPU model is expensive to keep loaded just in case someone calls it. In our GLM 5.3 test, starting from downloaded weights took 8m22s–8m29s, while a Snapshot restore answered in 33–34s. That avoided nearly eight minutes of initialization in the measured runs. If the model scales to zero, those eight GPUs can serve other workloads instead of sitting idle; a retained Snapshot provides a much faster path back when demand returns. The single-GPU models showed the same pattern: cold starts of 1m51s–2m44s became 6–9s after restore. In our view, this could change the economics of a broad model catalog and help add serving capacity quickly during bursts. These are worker-Pod-to-first-response measurements on an already-running node, not the full wait a client would experience when a GPU node must also be provisioned.
Where this fits in Saturn Cloud
Saturn Cloud’s Token Factory platform helps AI cloud operators turn GPU fleets into branded, customer-facing inference services. It integrates with NVIDIA DSX OS, the open, modular software portfolio of the NVIDIA DSX AI Factory Platform. NVIDIA Run:ai provides AI workload and GPU orchestration, and NVIDIA Dynamo provides distributed inference serving. Saturn Cloud adds model management, shared and dedicated service tiers, API access, per-token metering, and billing.
Snapshot addresses a complementary challenge: how quickly an initialized inference worker can return to service. These benchmarks explore how that capability could support Saturn Cloud’s scale-to-zero model catalog, helping operators use GPU capacity more efficiently while giving customers a shorter wait when an inactive model is requested.
Getting started
Snapshot provides Kubernetes components for capturing and restoring initialized inference workers. In this experiment, we captured a PodSnapshot from a ready vLLM source Pod and restored a compatible worker from that artifact. This is a performance experiment, not a Saturn Cloud deployment guide.
To explore how Saturn Cloud builds on NVIDIA DSX software to help you deliver branded, metered inference services on your own GPU infrastructure, contact us.
