Deploy DeepSeek V4-Pro on GPU Cloud: Single-Node 8x H200 Setup for the 1.6T MoE with vLLM (2026 Guide)

DeepSeek V4-Pro is the largest model in the DeepSeek V4 family: approximately 1.6 trillion total parameters, 49 billion active per forward pass, 384 routed experts, 1 million token context via DeepSeek V4's hybrid CSA/HCA sparse attention, and an MIT license. The headline parameter count makes it look like a multi-node model, but it isn't. The official checkpoint ships in native mixed precision, FP4 for the MoE experts and FP8 for everything else, and weighs about 865 GB on disk. A single 8x H200 SXM5 node has 1,128 GB of VRAM, which holds the weights with roughly 250 GB left for KV cache and runtime.

TL;DR: How to Deploy DeepSeek V4-Pro on GPU Cloud

• Weights: DeepSeek V4-Pro (1.6T total, 49B active) ships as an FP4-expert, FP8-elsewhere checkpoint of ~865 GB, not the ~1,600 GB a 1-byte-per-parameter estimate suggests.
• Minimum hardware: one 8x H200 141GB node (1,128 GB) fits it with ~250 GB for KV cache. 8x H100 80GB (640 GB) does not fit.
• Price: an 8x H200 node on Spheron H200 runs $44.08/hr ($5.51/hr per GPU) as of 06 Oct 2026.
• B200: 8x B200 (1,536 GB) fits the native checkpoint with ~670 GB of KV headroom, or NVIDIA's ~913 GB NVFP4 re-quant with ~620 GB.
• Multi-node: 2-3 H200 nodes only pay off for very long context or high concurrency, not for fitting weights.

This guide covers the single-node deployment path first, then the multi-node path for long-context KV budgets: VRAM math, tensor and pipeline parallelism configuration in vLLM, KV cache planning at 1M context, NVFP4 on B200, and live Spheron pricing for each configuration. For the same exercise on an even larger model, deploying Kimi K3 works through 2.8T parameters and a 1.56TB MXFP4 checkpoint.

If a smaller footprint is acceptable, start with the DeepSeek V4-Flash deployment guide. V4-Flash (284B total, 13B active) fits on 4x H100 SXM5 and delivers 3-5x higher throughput per dollar at the cost of lower accuracy on complex tasks. And before provisioning an 8x H200 node at all, run your monthly token volume through the DeepSeek API vs self-hosted cost and privacy comparison to confirm self-hosting is the cheaper path at your scale. If your reasoning workload doesn't need V4-Pro's full 1.6T footprint, Arcee's 400B Trinity-Large-Thinking is a smaller Apache 2.0 reasoning MoE that fits on a single 8x H100 node at FP8, and the Trinity-Large-Thinking hardware sizing guide covers the VRAM math and a direct cost comparison against Claude Opus 4.6.

DeepSeek V4-Pro vs V4-Flash

Both models fit on a single node. The difference with V4-Pro is which node: at ~865 GB of weights it needs 141GB H200s or larger, and an 8x H100 80GB node (640 GB) falls short by more than 200 GB before any KV cache is allocated. Pipeline parallelism across nodes is an option for extra KV headroom, not a requirement for loading the model.

The reason all weights must stay in VRAM is specific to MoE architecture: the router dispatches any token to any of the 384 experts on any forward pass. Unlike a dense model where you can shard weights across GPUs and reconstruct via tensor parallelism, every expert weight must be resident in fast GPU memory at inference time. For more on why active parameter count does not reduce VRAM requirements, see the MoE inference optimization guide.

GPU and Node Requirements for V4-Pro

VRAM Math

The 1-byte-per-parameter shortcut gives ~1,600 GB, which is why V4-Pro is often sized as a two-node model. That figure only applies if you upcast the FP4 experts to FP8, and there is no reason to do that for serving. Use the checkpoint as shipped.

Node Configurations

KV headroom here is total VRAM minus ~865 GB of weights, before vLLM's own runtime overhead, so treat it as an upper bound.

Critical clarification: the 49B active parameter count tells you about compute speed, not memory. All 384 expert weight sets must be in VRAM because the router can select any expert at any layer on any token. There is no way to predict ahead of time which experts will be skipped. This is why 49B active parameters does not simplify the VRAM budget at all.

For VRAM planning tools, the GPU memory requirements guide covers the full formula including KV cache and activation overhead.

Multi-Node Architecture: When You Need It, and How It Works

A single 8x H200 node is the right default. Go multi-node only when the ~250 GB of KV headroom on that node cannot hold the context length and concurrency your traffic needs. The rest of this section applies to that case.

Parallelism Axes

Running V4-Pro across multiple nodes combines three parallelism strategies:

Tensor parallelism (TP) splits each attention and MLP layer across GPUs within a single node via NVLink. For V4-Pro, TP=8 is the standard choice for an 8-GPU node, on one node or many.

Pipeline parallelism (PP) splits the model's transformer blocks across nodes. With PP=2 across two nodes, node 0 handles the first half of the layers and node 1 handles the second half. Tokens flow from node 0 to node 1 in sequence. With PP=3 across three nodes, the model is split into thirds. Each node then holds a smaller share of the weights, which frees more VRAM on every node for KV cache.

Expert parallelism (EP) distributes the MoE expert sets across GPUs within the PP stage assigned to each node. Within each node, expert routing happens over NVLink at sub-microsecond latency. This is the fast path.

Why Cross-Node Costs More

The problem with cross-node is latency. When the pipeline stage on node 0 finishes processing a batch and needs to pass activations to node 1, that communication travels over InfiniBand or RoCE (40-100 microseconds per hop) instead of NVLink (under 1 microsecond within a node). Pipeline parallelism structures the model so that expert groups stay co-located within each node's PP stage, minimizing the frequency of cross-node token dispatch. That overhead is the reason to stay on one node when the KV budget allows it.

For background on InfiniBand vs RoCE tradeoffs, the GPU networking InfiniBand vs RoCE guide has the latency benchmarks and cost math that apply directly to this workload.

Network Requirements

InfiniBand NDR 400G per node is the production standard for multi-node MoE serving. RoCEv2 at 400G works but adds 10-20% throughput overhead from higher congestion-control latency. For bare-metal cloud deployments, confirm the interconnect type before provisioning: ConnectX-7 or ConnectX-8 NICs and a rail-optimized fabric. Spheron multi-node clusters include InfiniBand or high-speed RoCE; check the instance spec for the specific fabric before provisioning.

Step-by-Step: vLLM for V4-Pro

Prerequisites

• vLLM v0.19+ (same version required across all nodes if you go multi-node)
• CUDA 12.4+, Python 3.10+
• NVLink within each node; InfiniBand or high-speed RoCE between nodes for multi-node
• Fast local NVMe or a persistent volume for weights (shared NFS or replicated weights for multi-node)
• HuggingFace credentials for deepseek-ai/DeepSeek-V4-Pro
• Ray installed on all nodes (multi-node only)

Download Weights

huggingface-cli download deepseek-ai/DeepSeek-V4-Pro \
  --local-dir /mnt/models/deepseek-v4-pro

The official checkpoint is approximately 865 GB. Use a persistent storage volume to avoid re-downloading on restarts. If you have multiple nodes, either mount the same NFS volume on all nodes or replicate the weights to fast local NVMe on each node before starting.

Single-Node Setup (8x H200 SXM5, Recommended)

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

There is no --quantization flag here: vLLM reads the FP4/FP8 quantization config from the checkpoint. With ~250 GB of KV headroom, 128K tokens is comfortable for a small number of concurrent sessions. No Ray cluster or pipeline parallelism needed.

Two-Node Setup (16x H200 SXM5, Long Context)

# Node 0: start Ray head
ray start --head --port=6379

# Node 1: connect to Ray head
ray start --address='<node0-ip>:6379'

# Node 0: launch vLLM across both nodes
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 2 \
  --enable-expert-parallel \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 524288 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

With two nodes you get ~1,390 GB of KV headroom after weights. That supports 512K-token requests at low concurrency, or many more 128K sessions than one node can hold.

Three-Node Setup (24x H200 SXM5, Full 1M Context)

# Node 0: start Ray head
ray start --head --port=6379

# Node 1 and Node 2: connect
ray start --address='<node0-ip>:6379'

# Node 0: launch with PP=3 and full 1M context
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 3 \
  --enable-expert-parallel \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 1048576 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

Set NCCL environment variables before launching either multi-node setup to enable InfiniBand RDMA:

export NCCL_IB_DISABLE=0
export NCCL_IB_HCA=mlx5_0:1
export NCCL_IB_GID_INDEX=3
export NCCL_DEBUG=WARN
export NCCL_NET_GDR_READ=1

Single-Node Setup (8x B200 SXM6)

An 8x B200 SXM6 node has 1,536 GB of VRAM. The official ~865 GB checkpoint fits with roughly 670 GB for KV cache using the same single-node command as H200. NVIDIA's NVFP4 re-quant (~913 GB) fits with roughly 620 GB and runs the experts on Blackwell's hardware FP4 path:

vllm serve nvidia/DeepSeek-V4-Pro-NVFP4 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --quantization modelopt_fp4 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 524288 \
  --gpu-memory-utilization 0.92 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

Replace the placeholder with the NVFP4 repo name from NVIDIA's Hugging Face organization. NVFP4 requires vLLM v0.19+ and B200 with appropriate driver support. If your vLLM build does not yet support --quantization modelopt_fp4, check the vLLM changelog for the minimum version with B200 NVFP4 serving enabled, or serve the official checkpoint instead.

SGLang Alternative

SGLang also supports DeepSeek V4 with tensor parallelism on a single node:

python -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Pro \
  --tp 8 \
  --context-length 131072

For multi-node SGLang with pipeline parallelism, see the SGLang multi-node documentation.

Test the Endpoint

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[{"role": "user", "content": "Write a Python function to solve the N-queens problem."}],
    max_tokens=512,
)
print(response.choices[0].message.content)

Memory Planning at 1M Context

KV cache memory grows linearly with context length for V4-Pro (the same sparse attention mechanism that enables 1M context keeps active KV compute bounded, but all selected KV entries must still reside in VRAM). This is the budget that decides whether you need more than one node.

Session counts divide the KV headroom from the node table (~250 GB, ~1,390 GB and ~2,520 GB) by the per-session estimate, and assume FP8 KV cache (--kv-cache-dtype fp8_e5m2). BF16 KV cache doubles the per-session figure. Real numbers vary with head dimension, layer count and runtime overhead, so measure on your own traffic.

Set --max-model-len to your actual traffic max, not 1M by default. If real requests cap at 64K tokens, set --max-model-len 65536 and a single 8x H200 node goes further than the table suggests. The KV cache optimization guide covers per-session KV budgeting in more detail.

Precision Options and Node Count

The official checkpoint is the default recommendation on both H200 and B200. It is already 4-bit where most of the parameters live, so there is little memory left to win by quantizing further, and upcasting the experts to FP8 only doubles their footprint and pushes you onto a second node for nothing.

NVFP4 on B200 is a compute choice, not a memory one. It is about 50 GB larger than the official checkpoint, but it targets Blackwell's native FP4 tensor cores. The full dynamic expert quantization background is in the dynamic expert quantization and MoE offloading guide.

INT4 schemes like AWQ buy almost nothing here: the experts are already FP4, and requantizing the FP8 attention and shared layers to INT4 saves a small fraction of the ~865 GB. It still will not fit an 8x H100 80GB node with usable KV headroom.

Node Cost on Spheron

For V4-Pro the practical pricing unit is the 8-GPU node bundle.

The single-node row is the one most teams should price. The multi-node rows only make sense if your KV budget from the table above demands them.

Cost-per-million-token math, once you have measured throughput on your own deployment:

Node $/hr / 3600 sec/hr = $/sec
$/sec / throughput tok/s = $/tok
$/tok * 1,000,000 = $/M tokens

Throughput depends heavily on batch size, context length and prompt mix, so benchmark your real traffic with benchmark_throughput.py before committing to a configuration.

Spot capacity can be reclaimed at any time without notice, so use it for workloads that can tolerate interruption. For sustained production serving, on-demand is the safer choice.

Compare these figures against V4-Flash: 4x H100 SXM5 ($10.56/hr) typically delivers 3-5x higher throughput per dollar than V4-Pro; check both setups against live rates. Flash is the cost-efficient default; V4-Pro is justified when quality on complex tasks is the deciding factor.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 06 Oct 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

When to Use V4-Pro vs V4-Flash vs Routing

For most production deployments, the practical answer is a routing layer: run V4-Flash for 90% of calls and escalate to V4-Pro for requests that trigger a complexity classifier. Task types that reliably benefit from Pro: chain-of-thought traces over 10 steps, multi-file code generation with dependency awareness, competitive programming problems at the hardest difficulty tier, and long-document synthesis where citation accuracy matters. Short agentic calls, single-turn chat, and summarization typically show no measurable quality gain with Pro.

This routing strategy also changes the spot calculus. V4-Flash spot interruptions during 30-second agentic calls lose almost no work. V4-Pro spot interruptions during 5-minute deep reasoning runs lose significant output. Reserve on-demand or reserved capacity for the Pro endpoint if you use this pattern.

If your complex-coding workload is specifically agentic software engineering rather than general reasoning, it's worth benchmarking against Z.ai's GLM-5.3 too. Setting up GLM-5.3 on GPU cloud covers a similarly sized MoE that jumped 6x on Terminal-Bench 3.0 over its predecessor, on the same single 8x H200 footprint as V4-Pro.

DeepSeek V4-Pro's ~865 GB checkpoint fits a single 8x H200 node, so most teams can serve it without a multi-node cluster. Spheron's H200 SXM5 nodes run it on one box, and include InfiniBand-grade interconnect when long-context workloads need two or three.

H200 SXM5 on Spheron | B200 SXM6 on Spheron | View all GPU pricing

Deploy DeepSeek V4-Pro on Spheron →