SGLang's documentation describes a feature called Paged Experts that changes a sizing question a lot of teams have been answering the expensive way: "how many GPUs does this mixture-of-experts checkpoint need," where the usual answer is one GPU per however many experts' worth of weights don't fit in VRAM. Paged Experts answers it differently: keep a handful of experts resident on the GPU, park the rest in host RAM, and page them in as the router asks for them. The flag is --enable-paged-experts, it's off by default, and it's restricted to a single GPU on purpose.
TL;DR: What SGLang Paged Experts Lets You Run on a Single GPU
SGLang's --enable-paged-experts keeps K routed experts resident on one GPU per layer, with the rest in pinned host RAM, so an MoE checkpoint bigger than VRAM runs on one card.
- Mechanism: compute stays on the GPU; only expert storage moves to host RAM, per SGLang's docs.
- Tested fit: Qwen3-30B-A3B-GPTQ-Int4 runs on a 16 GB GPU with K=63 of 128 experts resident at 208 tok/s.
- Extreme case: DeepSeek-V4-Flash's 137 GiB of experts served from a 16 GB RTX 5070 Ti plus 62 GB host RAM at roughly 14 tok/s.
- Limits: single GPU only; no speculative decoding or expert/tensor parallelism.
- Buying consequence: Spheron's on-demand catalog covers these single-GPU tiers; H100 runs $2.65/hr as of 17 Nov 2026. Compare pricing.
What SGLang Paged Experts Actually Does
Paged Experts serves an MoE model whose expert weights don't fit in GPU memory by keeping K of E routed experts resident in a GPU slot table per layer, with all E experts held in pinned host memory; whenever a token's router selects an expert that isn't currently in a slot, SGLang copies it in before the expert GEMM runs. The model's actual computation, every matrix multiply, every attention step, still runs on the GPU exactly as it would without the flag. What moves is storage, not compute, and that distinction is what separates Paged Experts from --cpu-offload-gb and CPU expert offload, where the offloaded experts' math runs on the CPU itself.
This works for MoE specifically, and not dense models, because a router only touches a handful of experts per token. Our guide to MoE inference memory sizing covers why total parameter count, not active parameter count, sets your VRAM floor on a normal deployment: you still have to hold every expert in memory even though each token only activates a few of them. Paged Experts stops doing that on the GPU specifically, because the sparsity that makes MoE cheap to compute also makes it cheap to page: most experts sit idle most of the time, which is exactly the access pattern host-memory paging is good at serving.
How the K-Slot Resident Table and Host-Memory Paging Work
Each decode step collects the distinct experts the current batch of tokens actually routed to, then splits them into waves of at most K, with already-resident experts scheduled first. A wave that needs an expert not currently in a slot evicts the least-recently-used resident expert, pages the missing one in from host memory, runs the expert GEMM, and merges that wave's output with every other wave's before the model computes its final top-k weighted sum, so the result doesn't depend on how SGLang happened to split the waves. When tokens times the router's top_k fits inside K, the whole operation resolves in one wave and gets captured inside SGLang's decode CUDA graph; larger batches run without a graph, and prefill plans its waves on the host CPU, running paged MoE layers as an eager break under breakable prefill CUDA graphs.
K is therefore a latency knob with a discontinuity in it, not just a memory knob. Stay under the one-wave threshold and you keep graph-captured decode; go over it and you lose that optimization on top of paying for more page-ins, which is worth knowing before picking K from a VRAM budget alone.
Paged Experts vs KV Cache Offloading: The Memory You're Trading
KV cache offloading and expert paging solve two different memory problems that happen to compete for the same GPU budget. KV cache offloading moves a request's growing attention cache, the thing that scales with context length and concurrency, out to host memory or disk so more sequences fit at once; it applies to any transformer, dense or MoE, and it's the feature most "host-memory offloading" writeups on vLLM and similar stacks are actually describing. Paged Experts moves a slice of the model's own weights instead, the routed expert matrices that make an MoE layer an MoE layer. One scales with how many requests you're serving and how long they run; the other scales with how big the checkpoint is and how many of its experts you can afford to keep warm.
SGLang doesn't let you dodge that collision by running both at max settings. In Paged Experts' default auto mode, SGLang sizes the KV cache first, using whatever combination of --context-length, --max-running-requests, or --max-total-tokens you've set, and hands whatever static memory is left over to resident expert slots. A longer context window means less room for K, and a smaller K means more page-ins per step: it's not VRAM versus nothing, it's KV cache budget versus resident-expert budget, carved out of the same static memory pool.
Dynamic expert quantization is a useful contrast, since it solves an adjacent problem with a different lever: it keeps every expert resident but compresses the cold ones to a lower precision, trading accuracy headroom for memory instead of trading memory for page-in latency. Paged Experts keeps every expert at full precision and trades latency instead.
Picking K: --paged-experts-num-resident and What SGLang's Own Numbers Show
--paged-experts-num-resident sets K, the number of routed experts SGLang keeps resident per layer, and it must be at least your router's top_k; the default is auto, which derives K from whatever static memory the KV cache sizing above left behind. Auto mode can't be combined with --enable-memory-saver, --enable-lora, --enable-nvidia-megakernel, or post-capture KV sizing, so any of those require setting K explicitly.
SGLang's own documentation publishes a worked example on Qwen3-30B-A3B-GPTQ-Int4, a 128-expert model at 2.43 MB per expert and 117 MB per resident slot across 48 MoE layers, run on a 16 GB GPU at --mem-fraction-static 0.88. The default 40,960-token context leaves room for K=63 of 128 experts resident, at 208 tok/s decode; dropping to --context-length 16384 frees up enough memory for K=86, at 231 tok/s. SGLang's own numbers show decode speed flattening from roughly K=88 upward, meaning past a certain point you're giving up context length for page-ins you were already avoiding.
For checkpoints too large for host RAM, --paged-experts-enable-disk and --paged-experts-host-memory-gb add an on-disk expert cache tier; without the disk flag, startup fails outright if the experts don't fit in your configured host memory budget. SGLang's docs give a genuinely extreme case: DeepSeek-V4-Flash, 137 GiB of total expert weights, served on a 16 GB RTX 5070 Ti with 62 GB of system RAM and a 45 GB host memory budget, using the disk tier over PCIe 5.0 NVMe. Decode ran at roughly 14 tok/s, and prefilling a 1.5k-token prompt took about 8.5 seconds, with the disk cache itself coming to 13.9 GiB for the GPTQ-Int4 variant of Qwen3-30B-A3B on the same machine. Worth flagging separately: SGLang's SSD Expert Pack is a different feature with its own NVMe I/O path and layout, not the --paged-experts-enable-disk tier described here, so don't confuse the two when you're reading SGLang's changelogs.
What Breaks: Single GPU Only, No Spec Decoding, No EP
Paged Experts is deliberately a single-GPU feature. SGLang rejects tensor, expert, pipeline, and data parallelism outright when the flag is set, along with EPLB, fused shared experts (unless you pass --disable-shared-experts-fusion), and --enable-fused-moe-sum-all-reduce. If your plan was ever to scale a paged deployment across GPUs, it doesn't work; EPLB and wide expert parallelism is the right feature once you have more than one GPU to spread experts across. Speculative decoding is also rejected at startup, so a paged deployment can't lean on draft-model acceleration to make up the page-in latency.
Quantization support is split by runner. The Triton runner covers unquantized bf16/fp16, block FP8/INT8, and W8A8 FP8/INT8; the Marlin runner covers GPTQ, AWQ, NVFP4, and MXFP4. Anything outside those two lists is rejected, and so are the dummy and gguf load formats. On accuracy: under the Triton runner with unquantized, FP8, or INT8 experts, paged output is bit-identical to unpaged output when run with --enable-deterministic-inference; Marlin-runner quantized formats can differ slightly, within Marlin's normal batch-shape and atomic-add noise. SGLang's own GSM8K numbers on gpt-oss-20b in MXFP4, 200 questions, with 16 of 32 experts resident per layer, came in at 0.96 to 0.97 accuracy against 0.975 unpaged, a one to two point gap for running on a fraction of the resident expert memory a full load would need. Checkpoints also have to load as one mapped safetensors tensor per expert, so gpt-oss-style stacked expert tensors are refused outright.
The Buying Consequence: Rent One Smaller GPU Instead of a Multi-GPU Node
Everything above adds up to one practical decision: size K against your context length and your model's total expert memory, and "this checkpoint needs four or eight GPUs" turns into "this checkpoint needs one GPU and enough host RAM." That's a procurement decision, not just a performance tweak, because a single-GPU instance and a multi-GPU node aren't priced or provisioned the same way.
Spheron's on-demand catalog lists single GPUs from RTX 6000 Ada through B300, including RTX PRO 6000, RTX 5090, RTX 4090, L40S, A100, H100, H200, B200, and MI300X, billed per minute after a 20-minute minimum with no contract required, per Spheron's pricing page. That's the exact single-GPU range a Paged Experts deployment targets instead of a multi-GPU node:
| GPU | On-demand rate | Fits |
|---|---|---|
| RTX PRO 6000 | $2.39/hr | 24-31B-class MoE checkpoints with resident K tuned down for smaller VRAM |
| H100 | $2.65/hr | Larger MoE checkpoints where K needs more VRAM headroom for context length |
| H200 | $5.52/hr | Checkpoints pushing K higher without sacrificing context, or paired with a larger host-RAM instance for the disk tier |
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 17 Nov 2026; check current GPU pricing → for live rates before provisioning.
Per-minute billing with no contract matches how you'll actually tune this feature: picking K and a host memory budget is trial and error against your own context length and quantization format, not a number you get right on the first try. Spot is available on the same tiers for lower-stakes offline or batch runs, with the caveat that spot capacity can be reclaimed without notice, which matters less here since Paged Experts' single-GPU, no-speculative-decoding profile was never built for latency-critical serving anyway.
Two honest limits before you rent anything. Spheron's pricing page describes GPU rental terms, not a published host-RAM-per-instance spec, so confirm an instance's system RAM actually covers your model's total expert size, or budget for the disk tier, before you provision. And none of this helps if you need real throughput or concurrency: Paged Experts rejects EPLB and every form of parallelism, so a workload that's VRAM-bound because of traffic, not checkpoint size, is still better served by Spheron's multi-GPU or reserved NVLink instances. For a worked example of the kind of model this feature targets at larger scale, see our Qwen3.8 Max GPU sizing guide; SGLang's own examples here use the smaller Qwen3-30B-A3B family as the baseline case.
Sizing K against your context length is what decides whether a giant MoE checkpoint needs a single rented GPU or a multi-GPU node. Spheron's on-demand catalog covers the single-GPU tiers this trade targets, billed per minute with no contract.
Rent an H100 on Spheron → | Check H200 availability → | View all GPU pricing →
Frequently Asked Questions
Paged Experts is an SGLang flag, --enable-paged-experts, that lets a mixture-of-experts model run when its total expert weights don't fit in one GPU's VRAM. It keeps K routed experts resident in a GPU slot table per layer, holds the full expert set in pinned host RAM, and copies missing experts into a slot before each expert GEMM runs. The model's computation stays entirely on the GPU; only expert storage is split across tiers.
KV cache offloading moves a request's growing attention cache to host or disk so more concurrent sequences fit, and it applies to any transformer. Paging experts to host memory moves a fraction of the model's own weights instead, specifically the routed expert matrices in an MoE layer, so a checkpoint whose experts exceed VRAM can still load on one GPU. SGLang's auto mode even sizes the KV cache first and hands the GPU memory left over to resident expert slots, so the two features compete for the same budget rather than solving the same problem.
Yes, with --enable-paged-experts, as long as the GPU can hold K resident experts per layer plus the KV cache and the host has enough pinned RAM for every expert. For checkpoints whose total expert weights exceed host RAM too, --paged-experts-enable-disk adds an on-disk cache so startup doesn't fail, at the cost of decode steps that may wait up to 2 seconds for a disk read.
Speed drops because missing experts have to be copied in over PCIe before their GEMM can run, and that cost scales with how many of the model's experts actually fit resident. Accuracy is close to unaffected on the Triton runner: unquantized, FP8, and INT8 paged experts return bit-identical output to unpaged runs when --enable-deterministic-inference is set, per SGLang's own documentation. On gpt-oss-20b in MXFP4 with 16 of 32 experts resident per layer, paged GSM8K accuracy came in at 0.96 to 0.97 against 0.975 unpaged, roughly a one to two point gap.
Whatever single GPU holds your K resident experts per layer plus a usable KV cache at your target context length, since Paged Experts rejects tensor, expert, pipeline, and data parallelism outright. A 16 GB card handled Qwen3-30B-A3B-GPTQ-Int4 in SGLang's own published numbers; a 16 GB RTX 5070 Ti with 62 GB of host RAM served DeepSeek-V4-Flash's 137 GiB of expert weights using the disk tier. Spheron's on-demand catalog covers that single-GPU range from RTX 6000 Ada through B300, so the sizing decision is which single card, not which cluster.







