Deploy Kimi K3 on GPU Cloud: VRAM and Cluster Sizing Guide (2026)

Moonshot AI published Kimi K3's open weights on July 26, 2026, a day ahead of the July 27 target it had set for itself. The model card repeats the line every giant open MoE release ships with: run it on a "supernode configuration with 64 or more accelerators." That's not a cluster-sizing plan, it's a footnote. Here's the actual arithmetic: 2.8 trillion total parameters, three quantization tiers with three very different VRAM footprints, and the concrete B300, B200, and GB200 NVL72 configs that get you from zero to a running vLLM endpoint.

If you're coming from Kimi K2.7 Code or an earlier release (K2.6, K2.5), the jump to K3 isn't incremental. Total parameter count nearly triples, from 1T to 2.8T, and the deployment math changes with it. If you haven't committed to a model yet and want the broader self-hosting landscape first, the open-source LLM VRAM guide is worth reading before you provision a cluster for a 2.8T-parameter model.

What Kimi K3 Actually Is: 2.8T Total Params, 16 of 896 Active Experts

Kimi K3 is Moonshot AI's third-generation flagship: a 2.8 trillion parameter Mixture-of-Experts model with 896 total experts, 16 active per token, and a native 1,048,576-token (1M) context window. It launched through the Kimi app, Playground, and API on July 16, 2026, with public weights following on Hugging Face ten days later.

Moonshot's own launch comparison table puts K3 at 77.8 on Program Bench, 88.3 on Terminal Bench 2.1, and 93.5 on GPQA-Diamond, ahead of the Claude Opus 4.8 figures it lists alongside them (71.9, 84.6, and 91.0 on the same three). Treat those as self-reported vendor numbers, not independently verified results: Moonshot ran both models on its own harness, and independent trackers report a different picture for at least one of the three. On GPQA-Diamond specifically, third-party tracking sites have logged Opus 4.8 scores closer to 93-94%, which would make that metric a toss-up or a narrow Opus edge rather than the clear K3 win Moonshot's table shows. No independent, apples-to-apples run of all three benchmarks across both models is publicly available as of this post's publication date. For a wider view of where K3 lands against the rest of the current open-weight field, see the open-weight frontier model showdown, and weigh any single vendor table accordingly. None of that changes the deployment math below: a bigger benchmark gap doesn't shrink the weight footprint you have to fit in VRAM.

Kimi Delta Attention (KDA), AttnRes, and Stable LatentMoE Routing

Three architectural changes separate K3 from the K2 family, and two of them matter directly for deployment.

Kimi Delta Attention (KDA) is a hybrid linear attention mechanism built to keep attention cost bounded as context grows toward 1M tokens, rather than scaling the way standard multi-head attention does. Attention Residuals (AttnRes) works alongside it, selectively retrieving representations across depth instead of accumulating them uniformly through every layer. Combined, Moonshot reports these two changes deliver roughly 2.5x better overall scaling efficiency than Kimi K2.

Expert routing uses what Moonshot calls a "Stable LatentMoE" framework with quantile balancing: a token routes to an expert whenever that expert's router score lands in the top quantile for the batch. The mechanism is deterministic and hyperparameter-free, and Moonshot's stated goal is even expert utilization with zero dead experts, reinforced by a Per-Head Muon optimizer and a Sigmoid Tanh Unit (SiTU) activation designed to prevent the dead-neuron pathology that tends to show up in rarely activated experts at this scale.

Why Active-Parameter Counts Vary Across Sources (32B-104B) and What to Trust

Ask five sources how many parameters K3 actually activates per token and you'll get five different numbers. A naive 2.8T x 16/896 calculation, the ratio of active to total experts applied to the full parameter count, lands around 50-60B. Artificial Analysis lists 104B on its model tracking page. Other trackers round closer to 32B, on the assumption K3 kept roughly the same active-parameter ratio as K2.7.

The honest answer: as of publication, Moonshot has not released an official active-parameter figure or the split between always-active parameters (attention, KDA, AttnRes, embeddings, shared layers) and routed-expert parameters. The 2.8T x 16/896 shortcut only estimates the routed-expert slice; it says nothing about the always-on compute every token pays for regardless of which 16 experts fire. Until Moonshot ships a technical report with the real split, treat any single-number active-parameter claim, this post's included, as an estimate rather than a spec.

For hardware sizing, the ambiguity barely matters. What has to fit in VRAM is every expert's weights, not just the 16 that happen to activate on a given token, because the router can send any token to any of the 896 experts on any forward pass. Active-parameter count tells you about compute speed. It tells you nothing about memory footprint. That distinction is the whole reason the VRAM math below runs on total parameters, not active ones, exactly as it does for DeepSeek V4-Pro's 1.6T MoE.

VRAM Math: MXFP4 vs MXFP8 vs BF16 Weight Footprint

Three numbers matter for K3's weight footprint, and all three fall out of the same arithmetic: 2.8 trillion parameters times bytes per parameter.

The 1.4TB Baseline (2.8T Params x 4-bit) and Why Moonshot Trained for It

MXFP4 isn't a post-hoc quantization Moonshot bolted on after training in BF16. K3 was trained with quantization-aware training applied from the SFT stage onward, specifically so the released MXFP4 weights with MXFP8 activations serve natively at 4-bit without a separate calibration or quantization pass on your end. That's a meaningfully different starting point than K2.7 Code, where INT4 (AWQ) was an optional checkpoint alongside a native BF16/FP8 release. For K3, MXFP4 is the default. For the mechanics of block-scaled 4-bit formats and how MXFP4 differs from NVIDIA's NVFP4 hardware implementation, see the MXFP4 microscaling quantization guide.

The practical upshot: 1.4TB is the number you plan around unless you have a specific reason to run at higher precision.

MXFP8 (~2.8TB) and BF16 (~5.6TB) for Teams That Need Full Precision

Moonshot's public release is MXFP4 weights with MXFP8 activations, not a separate BF16 master checkpoint. If your team needs BF16 or MXFP8 weights, for further fine-tuning, for an accuracy baseline to compare quantization drift against, or simply because your serving stack doesn't yet support native MXFP4, check Moonshot's Hugging Face repository for an additional precision release before assuming one exists. The 2.8TB and 5.6TB figures above are what you'd need if such a checkpoint (or an upcast at load time) is available; they aren't a confirmed second SKU as of this post's publication date.

KV Cache Overhead at 1M Context and Why It Changes Your GPU Count

KDA's hybrid linear attention design exists specifically to keep the attention-side memory and compute cost from scaling the way standard full attention does as context stretches toward 1M tokens. That's good news for KV cache planning in principle. In practice, Moonshot has not published the exact per-token KV cache byte cost for K3, so there's no clean formula to plug into a spreadsheet yet.

What you can plan around: K2.6 and K2.7 Code, running Multi-head Latent Attention at a 256K context ceiling, needed roughly 40-80GB of extra headroom beyond base weights for a handful of concurrent long-context sessions. K3's context window is 4x larger. Even with KDA's architecture working in your favor, budget generously, well over 100GB beyond raw weights if you plan to serve concurrent long-context requests, until real production numbers are public. This is exactly why the GPU count you pick isn't just "however many GPUs fit the weights."

Reference Cluster Configs on B300 and B200

No single H100 or B200 node holds the full K3 model at any precision, and neither does a single B300 node at MXFP8 or BF16. But at MXFP4, the format Moonshot actually shipped, a single 8-GPU B300 node clears the bar on its own. Moonshot has not published a validated production hardware matrix of its own, so here's what the arithmetic and the hardware actually support.

Why Blackwell's Native FP4 Tensor Cores Are the Better Match for MXFP4

K3 was trained with quantization-aware training so the released MXFP4 weights serve natively at 4-bit, no calibration pass required. That detail matters for GPU choice, not just VRAM math. Hopper-generation cards (H100, H200) ship a Transformer Engine with native FP8 support only; running MXFP4 weights on Hopper means either upcasting to FP8/BF16 for compute or leaning on software-emulated 4-bit kernels, neither of which is free. Blackwell (B200, B300) ships a second-generation Transformer Engine with native FP4 Tensor Cores that execute K3's checkpoint format directly. For a model Moonshot built specifically for 4-bit serving, B300 and B200 are the GPUs whose compute path actually matches the weights, not just whose memory happens to fit them.

Single 8x B300 Node (2,304GB) Clears MXFP4 With Room to Spare

B300 SXM6 ships 288GB of HBM3e per GPU; an 8-GPU node totals 2,304GB. Against the 1.4TB (1,400GB) MXFP4 weight floor, that leaves roughly 904GB for KV cache, activations, and concurrency headroom, on a single node, with no InfiniBand hop between GPUs and no Ray cluster to stand up. That's a meaningfully simpler deployment than anything that needs two nodes to clear the same floor, and it's the config to reach for first if B300 capacity is available.

8x B200 (1,536GB) Also Fits, With Less Room to Work With

8x B200 (192GB per GPU, 1,536GB total) technically clears the 1.4TB MXFP4 floor too, but with only about 136GB left over. That's workable for short-context smoke testing, not for anything close to a 1M-token production session. Scale to 16x B200 across two 8-GPU nodes (3,072GB total) and the headroom problem disappears, roughly 1,672GB after weights, using data-parallel-size 16 for the attention/KDA path combined with expert-parallel-size 16 to spread the 896 experts across the 16 GPUs (56 experts per GPU). That's the same DP16+EP16 topology vLLM's team validated ahead of K3's public weight release. The serving flags section below covers launch commands for both the single-node B300 config and the two-node B200 config.

GB200 NVL72 Slice Sizing for MXFP8/BF16 and Moonshot's "64+ Accelerators" Supernode Guidance

Once you move up to MXFP8 (~2.8TB) or BF16 (~5.6TB), even B300 node counts climb, two 8-GPU B300 nodes (16 GPUs, 4,608GB) clear MXFP8 with over 1.8TB of headroom, while BF16 needs closer to three B300 nodes. At that scale, a rack-scale system starts to make more sense than stacking standard nodes over InfiniBand.

GB200 NVL72 packs 72 Blackwell GPUs at 192GB HBM3e each on a single 130TB/s NVSwitch fabric, no InfiniBand hop between GPUs at all. Doing the bare weight-fit math: MXFP8's 2.8TB needs about 15 of those 72 GPUs; BF16's 5.6TB needs about 30. Both numbers are well under Moonshot's own guidance of "supernode configurations with 64 or more accelerators" for production serving. That gap between the bare weight-fit minimum (15-30 GPUs) and Moonshot's recommended floor (64+) is where KV cache headroom for 1M-context sessions, request concurrency, and redundancy actually live, not in the weights themselves. A full 72-GPU NVL72 rack clears the 64+ threshold comfortably, with the added benefit that GB200's native NVFP4 tensor core execution is hardware-compatible with K3's MXFP4 checkpoint format; see the NVFP4 vs MXFP4 guide for the format-level distinction if you're deciding between the two on Blackwell hardware.

GB200 NVL72 capacity on Spheron is reservation-based rather than listed at a live per-GPU on-demand rate; the GB200 NVL72 guide covers full rack specs, and GB200 NVL72 on Spheron is where you submit GPU count, timeline, and workload for a same-business-day availability check.

Spot vs On-Demand Tradeoffs for a Cluster This Size

Live Spheron pricing (fetched from the GPU offers API on 27 Jul 2026): B300 SXM6 on-demand starts at $9.16/hr per GPU, spot at $5.81/hr per GPU. B200 SXM6 on-demand runs $9.36/hr per GPU, spot $5.34/hr per GPU.

Spot is 37-43% cheaper across both GPU types, but the risk profile is different for a multi-GPU DP+EP cluster than it is for a single-GPU workload. Every GPU in a data-parallel + expert-parallel group holds a distinct shard of the 896 experts; reclaim any one of them and the whole serving group goes down, not just one replica. That risk applies even to the single-node 8x B300 config, since all 8 GPUs are still one DP+EP group. For a production endpoint serving live traffic, on-demand is the safer default. Spot is worth it for evals, offline batch generation, or any workload where you can tolerate the entire cluster restarting mid-run.

Pricing fluctuates based on GPU availability. The prices above are based on 27 Jul 2026 and may have changed. Check current GPU pricing → for live rates.

vLLM/SGLang Serving Flags for K3's Long-Context, Multimodal Requests

K3's weights landed less than 24 hours before this guide was written. Flag names below reflect vLLM's day-0 preview support and are likely to shift as the model moves from nightly builds to a stable release. Pin your vLLM commit hash in production and re-check the release notes before upgrading.

Tensor-Parallel, Expert-Parallel, and Data-Parallel Flags (Single-Node B300 Reference Config)

On a single 8-GPU B300 node, no Ray cluster or multi-node networking is required. Launch vLLM directly:

vllm serve moonshotai/Kimi-K3 \
    --data-parallel-size 8 \
    --enable-expert-parallel \
    --host 0.0.0.0 \
    --port 8000 \
    --tool-call-parser kimi_k2 \
    --reasoning-parser kimi_k2 \
    --max-model-len 1048576 \
    --trust-remote-code

--data-parallel-size 8 replicates the attention/KDA path across all 8 ranks, one per GPU. --enable-expert-parallel shards the 896 experts across those same 8 ranks, 112 experts per GPU. This is the same DP+EP pattern used for wide expert parallelism on other giant MoE models.

If you're deploying on two 8-GPU B200 nodes instead, start a Ray cluster spanning both nodes and double every DP/EP value to 16:

# Node 0: start Ray head
ray start --head --port=6379

# Node 1: connect to Ray head
ray start --address='<node0-ip>:6379'

# Node 0: launch vLLM across both nodes
vllm serve moonshotai/Kimi-K3 \
    --data-parallel-size 16 \
    --enable-expert-parallel \
    --host 0.0.0.0 \
    --port 8000 \
    --tool-call-parser kimi_k2 \
    --reasoning-parser kimi_k2 \
    --max-model-len 1048576 \
    --trust-remote-code

That 16-rank config, 56 experts per GPU, matches the exact topology vLLM's team validated for K3 ahead of the public weight drop.

For a comparison of when vLLM's DP+EP path versus SGLang's --ep-size/--enable-dp-attention combination makes more sense for your serving stack, see vLLM vs SGLang. The equivalent SGLang launch for a single B300 node:

python -m sglang.launch_server \
  --model-path moonshotai/Kimi-K3 \
  --tp 8 \
  --ep-size 8 \
  --enable-dp-attention \
  --dp 8 \
  --context-length 1048576 \
  --port 30000

KDA Prefill Caching and Why Standard Prefix Caching Doesn't Apply Yet

Standard prefix caching, the kind that gives K2-family models a free speedup on repeated system prompts and shared tool definitions in agentic workloads, works by hashing token-aligned KV blocks and reusing them across requests that share a prefix. KDA doesn't produce that kind of block-aligned KV state. Its hybrid linear attention mechanism maintains a compressed recurrent state per layer instead of discrete per-token key/value pairs, so a cache lookup built for standard attention doesn't decompose the same way against it.

vLLM shipped a KDA-specific caching implementation alongside K3's day-0 support to address this, but don't assume your existing prefix-cache hit-rate assumptions from K2.6 or K2.7 Code transfer directly. If your workload leans on long, repeated system prompts or tool schemas, benchmark actual cache hit rates on K3 before sizing concurrency around them.

Tool-Call-Parser, Reasoning-Parser, and Max-Model-Len for 1M Context

K3 uses the same kimi_k2 tool-call-parser and reasoning-parser as the K2 family, so existing MCP and function-calling integrations built against K2.6 or K2.7 Code should carry over with minimal changes. --max-model-len 1048576 sets the ceiling at the full 1M-token window, but set it to your actual traffic maximum, not the architectural ceiling by default. Every token of unused context length is KV cache headroom you're paying for and not using; if your real requests cap at 128K tokens, set --max-model-len 131072 and recover the difference for more concurrent sessions.

For multi-file or multi-document requests near the top of that range, add chunked prefill to avoid a memory spike on the first forward pass (shown here for the single-node B300 config; use --data-parallel-size 16 on a two-node B200 deployment):

vllm serve moonshotai/Kimi-K3 \
    --data-parallel-size 8 \
    --enable-expert-parallel \
    --enable-chunked-prefill \
    --max-num-batched-tokens 8192 \
    --tool-call-parser kimi_k2 \
    --reasoning-parser kimi_k2 \
    --max-model-len 1048576 \
    --trust-remote-code

MoonViT-V2 Vision Encoder Flags for Multimodal Requests

K3 ships with a MoonViT-V2 vision encoder, roughly 401M parameters, small enough that it doesn't meaningfully change the VRAM math above, but it does need its own flag to distribute encoding work across your GPU set. Following the same pattern K2.6 uses for the original MoonViT:

vllm serve moonshotai/Kimi-K3 \
    --data-parallel-size 8 \
    --enable-expert-parallel \
    --mm-encoder-tp-mode data \
    --tool-call-parser kimi_k2 \
    --reasoning-parser kimi_k2 \
    --max-model-len 1048576 \
    --trust-remote-code

Send multimodal requests through the standard OpenAI-compatible chat completions endpoint with image_url content blocks. Confirm the vision path is active by sending a test request with a base64-encoded image before routing production multimodal traffic to the endpoint. For provisioning and SSH setup, docs.spheron.ai covers connection steps for both single-node and multi-node clusters.

Kimi K3's 2.8T-parameter footprint means the cluster-sizing math matters more than the benchmark scores. Blackwell's native FP4 Tensor Cores are the right match for K3's MXFP4 weights, and a single 8-GPU Spheron B300 node fits the full model with headroom to spare.

Spheron B300 instances → | B200 GPU pricing → | View all GPU pricing →