vLLM Preload Weight Cache Daemon: Fast Restarts in vLLM 0.31 (2026)

vLLM 0.31 shipped a weight-cache daemon that changes what a model restart actually costs you. Instead of an engine reading gigabytes of weights off disk every time it starts, a standalone process holds the weights in GPU memory and hands a restarting engine a CUDA IPC handle to them. The 0.31.0 release that shipped it carried 717 commits from 307 contributors, 96 of them first-time contributors to the project.

TL;DR: What Is the vLLM Preload Weight Cache Daemon?

vLLM 0.31's weight-cache daemon keeps model weights resident in GPU memory, so restarts map them via CUDA IPC instead of reloading from disk.

• Mechanism: vllm preload --model <name> --tensor-parallel-size <n> starts it; engines restart with --load-format ipc_cache.
• Claimed gain: vLLM's docs say this cuts restart time "from minutes (disk load + quantization processing) to seconds."
• 0.31 adds: data-parallel engines and MTP/EAGLE draft models get their own daemon cache.
• Limit: zero_copy needs the daemon's GPU memory to stay live, covering an engine restart, not a GPU stop or spot reclaim.
• Spheron bills GPU instances per minute with no commitment, the backdrop for sizing allocated GPUs once restarts get cheap. Check current GPU pricing for live rates.

What vllm preload and the ipc_cache Load Format Actually Do

Run vllm preload --model <name> --tensor-parallel-size <n> and vLLM starts a dedicated per-GPU process, the WeightCacheDaemon, that loads one tensor-parallel shard of the named model into GPU memory and then sits there serving it, rather than exiting once the load finishes. Launch or restart a vLLM engine afterward with --load-format ipc_cache and it connects to that daemon over a Unix domain socket, receiving CUDA IPC handles to the already-resident weight tensors instead of reading the checkpoint files again.

You pick between two handoff modes through --model-loader-extra-config's mode key. The default, zero_copy, shares the daemon's existing GPU memory with the engine directly, so a restart adds no extra VRAM footprint for weights. copy mode clones the tensors into the engine's own memory instead, which costs a one-time copy but lets the daemon release its cache once the handoff completes. The rest of --model-loader-extra-config covers the production knobs: fallback (true by default) to drop back to a disk load if the daemon is unreachable, connect_timeout_s and state_timeout_s to bound the handshake, and socket_path/socket_dir to control where the Unix socket lives.

Before an engine maps a daemon's cached weights, vLLM runs a fingerprint check comparing checkpoint content, model architecture, tensor- and data-parallel configuration, dtype, quantization method, RoPE parameters, model revision, KV cache settings, and vLLM version itself. Any mismatch triggers the same disk-load fallback as an unreachable daemon, unless you explicitly disable it. That check is also why preload is a same-node, same-user feature: Unix domain sockets don't cross hosts, so the daemon and every engine it serves have to live on one machine. Preload supports tensor, expert, and data parallelism, including across nodes, but vLLM rejects pipeline parallelism outright when you try to launch the daemon.

vLLM Faster Model Reload: Measured Restart Time, Disk Load vs the Weight-Cache Daemon

The bottleneck the daemon targets is real and it's large. That weight-loading tax gets worse with contention. Before isolating per-worker CPU allocation and reusing saved kernel-tuning results, we ran DeepSeek V4.1 Flash on a fresh two-copy node that took 43.6 minutes to reach serving; 15.8 of those minutes came from vLLM handing every GPU worker 40 CPU threads instead of a fair share, and another 13 came from kernel autotuning that re-ran on every boot because the cache folder name changed each time. Fixing both, plus starting both copies together, cut that cold start to 11.9 minutes. None of that fix involved the weight-cache daemon, since it predates 0.31, but it shows the order of magnitude a disk-bound, CPU-contended load works against: tens of minutes, driven mostly by phases a resident, IPC-mapped weight cache skips entirely by never re-reading the checkpoint.

Keep this separate from a different restart-time change that shipped a release earlier. vLLM's 0.30 release notes credit Model Runner V2's process-level garbage-collection freezing with cutting engine initialization, not weight loading, from 28.9 seconds to 8.2 seconds on an H200, a figure vLLM reports for its own setup rather than an independent benchmark; see the MRV2 deployment guide for more on that separate change. The weight-cache daemon's seconds-not-minutes claim in 0.31 is specifically about the disk-to-VRAM weight transfer, the same transfer GPU Direct Storage speeds up for workloads that still have to read from disk every time; preload's answer is to avoid the re-read altogether for weights already loaded somewhere on the box.

Data Parallelism and MTP Draft Models in the Same 0.31 Release

Preload shipped in an earlier vLLM release scoped to tensor-parallel single-model serving. 0.31 extends it in two directions that matter once you're running anything past a single dense model on a single engine.

Data-parallel deployments can now preload through --data-parallel-size, --data-parallel-size-local, --data-parallel-start-rank, and --data-parallel-address, so a sharded-by-replica deployment gets the same IPC-mapped restart path as a plain tensor-parallel one. Speculative decoding gets its own daemon group: MTP, EAGLE, and EAGLE3 draft models run a second WeightCacheDaemon cluster on distinct *_draft.sock sockets, coordinated through a new --weight-cache-draft-master-port flag, so draft weights are cached and verified independently from the target model's. The MTP deployment guide covers the draft-head side of that pairing, and the same daemon-per-draft-group pattern applies to confidence-scheduled methods like DSpark.

Two more 0.31 additions are operational: a /health endpoint on the daemon, off by default until you pass --weight-cache-health-port, and a readiness-wait mechanism so an orchestrator can block routing traffic until every rank reports ready instead of guessing at a sleep timer. The daemon also runs FlashInfer's autotuning pass during preload by default and persists the results, so a later restart against the same daemon skips re-tuning too; the API reference exposes a config switch to turn that autotuning off if you'd rather skip it.

What CRIU Snapshots Hint at for Future Cold Starts

The same 0.31 release ships a second, unrelated restart-time feature at an earlier stage: experimental vllm snapshot create and vllm snapshot restore commands that use CRIU (Checkpoint/Restore In Userspace) to save and reload a fully initialized engine process, not just its weights. Where ipc_cache only caches weight tensors and needs a daemon running continuously to serve them, a CRIU snapshot captures the whole process state, weights, compiled CUDA kernels, and scheduler state, and restores it directly with no separate caching process needed.

The catch is scope. CRIU snapshot/restore currently works only on x86-64 Linux, a single NVIDIA GPU, and dense FP16 models at tensor-parallel size 1, which rules out most production multi-GPU serving today. The weight-cache daemon is what actually ships broadly in 0.31, across TP, expert, and data parallelism, and both CUDA and ROCm; CRIU snapshots are a preview of a restore path that could eventually cover compiled kernels and scheduler state too, not something to plan a deployment around yet.

Fewer Always-On GPUs: Sizing Autoscaled Inference After 0.31

If your engine restart takes minutes, keeping engines running through low traffic is the safe default, because spinning one back up is too slow to track demand. Once a restart takes seconds, that math changes: you can let an autoscaler cycle engines against real traffic instead of padding capacity to absorb a slow cold start, which is the lever that lets a fleet run fewer GPUs permanently allocated to idle engines.

That only applies to the engine process, not the GPU allocation underneath it. zero_copy mode needs the daemon's GPU memory to stay live for its IPC handles to remain valid, and it's explicitly incompatible with vLLM's sleep mode (CuMemAllocator offloading), so a daemon-backed engine doesn't survive the GPU itself being stopped or a spot instance being reclaimed, only the engine process restarting on a GPU still allocated to it. Spot capacity is worth planning around rather than treating as an edge case. The standard mitigation, running two copies of every model on separate nodes so a reclaim never takes a model fully offline, doesn't go away just because restarts got faster; it's a separate problem the daemon doesn't solve.

Where the daemon's math applies directly is crash recovery and traffic-driven cycling on GPUs you already hold. In that same load testing, we watched an engine crash restart on the same node in 3 minutes, the kind of recovery window a daemon-backed restart is built to compress toward seconds. For the GPU allocation itself, H100 SXM5 runs $2.64/hr on-demand and $2.20/hr spot as of 24 Oct 2026. That combination, cheap engine restarts on top of per-minute billing, is what makes "fewer always-on GPUs, cycled harder" workable, provided the spot-reclaim redundancy above still happens around it.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 24 Oct 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

Faster vLLM restarts change how many GPUs you need sitting idle between bursts of traffic, and per-minute billing is what makes acting on that math actually pay off.

Get started on Spheron →