Deploy Kimi K3.1 Pro on GPU Cloud: 2.8T MoE Setup Guide (2026)

Moonshot AI put Kimi K3.1 Pro's open weights on Hugging Face on 4 November 2026, and the first thing anyone planning to deploy Kimi K3.1 Pro on GPU cloud hardware needs to know is that it didn't reset the deployment math. The checkpoint is 1,561GB across 111 files, a near-twin of July's Kimi K3 release at 1,560.94GB, built on the same 2.8T-total, 104B-active-parameter architecture. Coverage of the release so far has been benchmark scores and API pricing, which is useful if you're deciding whether to call Moonshot's API, but not if you're deciding whether to self-host. This is the deployment guide: the real VRAM requirements, what minimum cluster actually fits, vLLM setup, the license limits that kick in once you serve it commercially, and what it costs to run yourself against calling the API instead.

TL;DR: What It Takes to Deploy Kimi K3.1 Pro on GPU Cloud

Deploying Kimi K3.1 Pro on GPU cloud needs a cluster sized for its 1.56TB checkpoint, not its 104B active-parameter count.

• Checkpoint: 1,561GB across 111 files on Hugging Face, almost identical to Kimi K3's 1,560.94GB MXFP4 checkpoint.
• Minimum cluster: one 8x B300 node (2,304GB HBM3e) fits it; four 8x H100 nodes work too, via a dequantization kernel.
• License: the new Kimi K3.1-Pro License requires a separate Moonshot agreement once trailing-12-month group revenue passes $20M and you resell inference or fine-tuning access.
• Price: Spheron runs 8x B300 at $68.08/hr spot as of 11 Nov 2026; on-demand B300, H200, and H100 tiers are also listed on current GPU pricing.
• Verdict: reuse your Kimi K3 cluster sizing almost as-is; the license is what actually changed.

Kimi K3.1 Pro: 2.8T Total, 104B Active, Same Architecture as K3

Kimi K3.1 Pro is a 2.8 trillion-parameter Mixture-of-Experts model that activates 104B parameters per token, with a 1,048,576-token (1M) context window. The weights ship natively in MXFP4 with MXFP8 activations, produced through quantization-aware training rather than a post-hoc quantization pass bolted on after the fact. That's the same training approach Kimi K3 used in July, and for good reason: K3.1 Pro inherits K3's architecture almost wholesale, 16 of 896 routed experts plus 2 always-on shared experts, the hybrid attention stack of 69 Kimi Delta Attention linear layers interleaved with 24 MLA layers across 93 total layers, and the MoonViT-V2 vision encoder for native multimodal input.

That continuity matters more than a headline parameter count. A 2.8T/104B-active model is a different machine to size than Moonshot's prior flagship, Kimi K2.7 Code, which runs 1T total parameters with only 32B active through 384 experts and fits comfortably on a single 8x H200 node. K3.1 Pro's jump to 896 experts and a checkpoint nearly triple the size is why the GPU math below looks nothing like K2.7 Code's, and almost identical to K3's.

One real change from K3: Moonshot has said it "identified and removed cyber-related data from both training stages" before publishing the open weights, deliberately weakening Kimi K3.1 Pro's hacking-related capabilities relative to what the closed model can do. That's a safety decision on Moonshot's part, not a deployment constraint, but it's worth knowing if you're benchmarking the open checkpoint against reports of the hosted version's behavior and seeing a gap.

On raw capability, Artificial Analysis's independent Intelligence Index puts Kimi K3.1 Pro at 46 against Claude Opus 5.5's 59, a real gap rather than the near-parity some early coverage suggested. That index is a third-party aggregate across many evals, not a single vendor's self-reported table, so it's a sturdier number than the kind of launch-day benchmark chart this blog flagged as provisional when K3 shipped against Opus 4.8's numbers on Moonshot's own harness. It's still one index, and the gap on any individual task, coding especially, can run narrower or wider than the aggregate suggests. Plan your cluster around the architecture, which doesn't change regardless of how any single score moves.

GPU and VRAM Requirements: Minimum Node Configuration for the 1.56TB Checkpoint

The published moonshotai/Kimi-K3.1-Pro repository is 1,561GB across 111 files (about 1.56TB), not the roughly 1.4TB a naive 2.8T x 0.5-byte MXFP4 estimate gives you, and not the 5.6TB BF16 figure that circulated in some early coverage: if you were planning around a 5.6TB download, you were planning against a number Moonshot never shipped. That real number is what decides whether a node loads the model or dies partway through weight loading.

Because the checkpoint is within half a gigabyte of Kimi K3's 1,560.94GB, the node math carries over directly:

A single B300 SXM6 node is the only one-node option, and it's the one to reach for first if the capacity is there: no multi-node bring-up, native FP4 tensor cores that execute the checkpoint's format directly, and roughly 743GB left over for KV cache and concurrency. 8x B200 clears the theoretical 1.4TB estimate with room to spare and still fails on the real checkpoint, which is exactly the trap that catches people who size from the parameter-count formula instead of the actual file size on Hugging Face.

On Hopper, the floor is four 8x H100 80GB nodes, 32 GPUs totaling 2,560GB. H100 has no native FP4 tensor cores, so serving MXFP4 weights there means dequantizing 4-bit weights to 16-bit inline during each matmul, the same marlin-style W4A16 approach Kimi K3's SGLang cookbook documents, and it leaves only a few gigabytes of KV cache per GPU once the per-rank duplication of attention, KDA, and vision-encoder weights is accounted for. H100 SXM5 instances are the cheaper and more widely available path onto this model; swap in H200 if you need real KV headroom at long context, since 141GB per GPU against the same 32-GPU topology turns that 3.6GB pool into closer to 90GB per GPU.

A comparably scaled model to check your intuition against is DeepSeek V4-Pro's 1.6T MoE, which fits on a single 8x H200 node because its checkpoint ships at roughly 865GB in mixed FP4/FP8 precision. Kimi K3.1 Pro's checkpoint is nearly double that, which is the whole reason it needs B300 or four H100 nodes rather than one.

Serving It: vLLM Setup and the License Limits That Affect Deployment

On a single 8-GPU B300 node, vLLM needs no Ray cluster or multi-node networking. A launch command that mirrors Kimi K3's day-0 recipe:

vllm serve moonshotai/Kimi-K3.1-Pro \
    --data-parallel-size 8 \
    --enable-expert-parallel \
    --host 0.0.0.0 \
    --port 8000 \
    --tool-call-parser kimi_k3 \
    --reasoning-parser kimi_k3 \
    --max-model-len 1048576 \
    --trust-remote-code

--data-parallel-size 8 replicates the attention and KDA path across all 8 ranks; --enable-expert-parallel shards the 896 experts across those same ranks. Because K3.1 Pro shares K3's architecture, the kimi_k3 parser names are the reasonable starting point, but verify them against vLLM's actual release notes for this model before you commit a production config. A checkpoint published days before this post was written is exactly the situation where flag names and parser support shift fast between a nightly build and a stable release, and K3's own launch carried the same caveat.

Weight-loading time matters more on a checkpoint this size than the benchmark score does, and the two biggest levers for cutting it aren't model-specific. On Spheron's own 8xB200 inference fleet, cutting DeepSeek V4.1 Flash's cold start from 43.6 minutes to 11.9 minutes (measured 16 Oct 2026, on a fresh two-copy node) came down to giving each vLLM GPU worker its actual share of the node's CPU threads instead of letting every worker claim 40, plus reusing the kernel-tuning cache across restarts instead of letting vLLM redo that tuning every boot because the cache folder name kept changing. Neither fix is Kimi-specific, and both apply directly to a four-node K3.1 Pro cluster, where weight loading already dominates the time to a serving endpoint. If you want a serving engine built specifically around the MLA-style attention layers Kimi models use, TokenSpeed's deployment guide covers the alternative to vLLM's default runner.

The part of this release that genuinely changed from K3 is the license. Kimi K3.1 Pro ships under a new Kimi K3.1-Pro License, not the Modified MIT license K3 used. If your "group revenue exceeds $20 million over any consecutive 12 months" and you're offering third-party inference or fine-tuning access on the model, you need a separate agreement with Moonshot before you can sell that access. Separately, the license requires any "commercial products with more than 100 million monthly active users, or more than $20 million in monthly revenue" to display "Kimi K3.1-Pro" prominently in its UI, a branding obligation rather than a payment one. Neither condition applies if you're using the model internally, or through Moonshot's own products or certified partners. For most teams self-hosting for internal inference, neither threshold is a practical concern; for anyone planning to resell access to a cluster running this model, it's a deployment decision to make before you provision, not after.

What It Costs to Run on Spheron vs the Kimi API

Self-hosting cost on Spheron scales with the cluster the table above requires, not with how many tokens you push through it. An 8x B300 node runs $68.08/hr spot and $12.42/hr per GPU on-demand as of 11 Nov 2026; the four-node H100 path runs $70.40/hr spot across 32 GPUs at the same snapshot. Spot and on-demand track different pools of partner capacity, so check both tokens above rather than assuming one is always the cheaper entry point. What should drive the choice on a cluster this size is reclaim risk, not price. On Spheron's own four-node 8xB200 inference fleet, a live spot-node kill under production load on 16 Oct 2026 finished in-flight streams on a second node in about 7 seconds with zero failures, and 16 of 16 in-progress tool calls still completed; that recovery works because Spheron runs every model as two copies on two separate nodes rather than one, so a reclaim never takes a model itself offline. A single 8x B300 node running Kimi K3.1 Pro has none of that redundancy: losing it mid-run means re-forming the whole data-parallel group and reloading 1.56TB of weights from cache, which is tolerable for evals and benchmarking runs and a real problem for production traffic unless you provision a second B300 node specifically to avoid it.

Moonshot also sells Kimi K3.1 Pro through its own API, the straightforward alternative to standing up a cluster at all, and the one most teams should default to until their request volume justifies the fixed cost of a dedicated node. Moonshot's listed rate is $2.00 per 1M input tokens and $10.00 per 1M output tokens, roughly half what Anthropic charges for Claude Opus 5.5 at $4.00 per 1M input tokens and $20.00 per 1M output tokens, current as of this post's date and worth rechecking before you budget against it, since both vendors adjust API pricing without much notice. The crossover point is the same calculation this blog has made for every API-only or dual-availability release: once sustained throughput needs outgrow what a per-token bill costs at your actual request volume, a reserved self-hosted cluster starts winning on unit economics, and K3.1 Pro's 104B active parameters per token make that crossover arrive sooner than it would for a smaller model. If you want the mirror case, where self-hosting isn't an option at all because no open weights exist, Qwen3.7 Max is the closed-weight counterpoint: there, the API is the only path, full stop. With K3.1 Pro, self-hosting is a real choice, and the license's $20M revenue threshold is the only scenario where Moonshot's terms, rather than your GPU bill, decide whether you can actually ship it.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 11 Nov 2026; other figures reflect the most recently published rates and may have changed. Check current GPU pricing → for live rates.

Kimi K3.1 Pro's deployment math is K3's math with a new license on top: the same single B300 node or four-node H100 cluster fits the checkpoint, and the $20M revenue threshold is the one new thing to check before you serve it commercially.

Check B300 availability → | H100 on Spheron → | View all GPU pricing →