Deploy DeepSeek V4-Pro on GPU Cloud: Multi-Node Setup for the 1.6T MoE with vLLM (2026 Guide)

DeepSeek V4-Pro is the largest model in the DeepSeek V4 family: approximately 1.6 trillion total parameters, 49 billion active per forward pass, 384 routed experts, and 1 million token context via DeepSeek V4's hybrid CSA/HCA sparse attention. At FP8 precision (1 byte per parameter), the full weight set is roughly 1,600 GB. A single 8x H200 SXM5 node tops out at 1,128 GB, which falls short by nearly 500 GB. Single-node deployment is not viable at FP8. Multi-node is the only path.

This guide covers the multi-node deployment path: node count math, pipeline parallelism configuration in vLLM, KV cache planning at 1M context, NVFP4 quantization on B200 for single-node serving, and live Spheron pricing for each configuration.

If a smaller footprint is acceptable, start with the DeepSeek V4-Flash deployment guide. V4-Flash (284B total, 13B active) fits on 4x H100 SXM5 at FP8 and delivers 3-5x higher throughput per dollar at the cost of lower accuracy on complex tasks. And before provisioning a 12-GPU cluster at all, run your monthly token volume through the DeepSeek API vs self-hosted cost and privacy comparison to confirm self-hosting is the cheaper path at your scale.

DeepSeek V4-Pro vs V4 vs V4-Flash

V4-Pro is in a different weight class from V4 and V4-Flash. V4 and Flash both fit on a single 8-GPU node. V4-Pro does not. The 1.6T weight footprint exceeds single-node VRAM on every current H100 and H200 configuration, which means pipeline parallelism across nodes is a hard requirement rather than an optimization.

The reason all weights must stay in VRAM is specific to MoE architecture: the router dispatches any token to any of the 384 experts on any forward pass. Unlike a dense model where you can shard weights across GPUs and reconstruct via tensor parallelism, every expert weight must be resident in fast GPU memory at inference time. For more on why active parameter count does not reduce VRAM requirements, see the MoE inference optimization guide.

GPU and Node Requirements for V4-Pro

VRAM Math

Node Configurations

Critical clarification: the 49B active parameter count tells you about compute speed, not memory. All 384 expert weight sets must be in VRAM because the router can select any expert at any layer on any token. There is no way to predict ahead of time which experts will be skipped. This is why 49B active parameters does not simplify the VRAM budget at all.

For VRAM planning tools, the GPU memory requirements guide covers the full formula including KV cache and activation overhead.

Multi-Node Architecture: Pipeline Parallelism, Expert Parallelism, and Interconnect

Parallelism Axes

Running V4-Pro across multiple nodes combines two parallelism strategies:

Tensor parallelism (TP) splits each attention and MLP layer across GPUs within a single node via NVLink. For V4-Pro, TP=8 is the standard choice for an 8-GPU node. This handles the attention layers, which are too large for a single GPU even after splitting the expert layers.

Pipeline parallelism (PP) splits the model's transformer blocks across nodes. With PP=2 across two nodes, node 0 handles layers 1-30 and node 1 handles layers 31-60. Tokens flow from node 0 to node 1 in sequence. With PP=3 across three nodes, the model is split into thirds. Pipeline parallelism is the key mechanism that makes multi-node V4-Pro feasible.

Expert parallelism (EP) distributes the MoE expert sets across GPUs within the PP stage assigned to each node. Within each node, expert routing happens over NVLink at sub-microsecond latency. This is the fast path.

Why Cross-Node Costs More

The problem with cross-node is latency. When the pipeline stage on node 0 finishes processing a batch and needs to pass activations to node 1, that communication travels over InfiniBand or RoCE (40-100 microseconds per hop) instead of NVLink (under 1 microsecond within a node). Pipeline parallelism structures the model so that expert groups stay co-located within each node's PP stage, minimizing the frequency of cross-node token dispatch.

For background on InfiniBand vs RoCE tradeoffs, the GPU networking InfiniBand vs RoCE guide has the latency benchmarks and cost math that apply directly to this workload.

Network Requirements

InfiniBand NDR 400G per node is the production standard for multi-node MoE serving. RoCEv2 at 400G works but adds 10-20% throughput overhead from higher congestion-control latency. For bare-metal cloud deployments, confirm the interconnect type before provisioning: ConnectX-7 or ConnectX-8 NICs and a rail-optimized fabric. Spheron multi-node clusters include InfiniBand or high-speed RoCE; check the instance spec for the specific fabric before provisioning.

Step-by-Step: Multi-Node vLLM for V4-Pro

Prerequisites

• vLLM v0.19+ on each node (same version required across all nodes)
• CUDA 12.4+, Python 3.10+
• NVLink within each node, InfiniBand or high-speed RoCE between nodes
• Shared NFS or replicated weights on each node
• HuggingFace credentials for deepseek-ai/DeepSeek-V4-Pro
• Ray installed on all nodes

Download Weights

huggingface-cli download deepseek-ai/DeepSeek-V4-Pro \
  --local-dir /mnt/models/deepseek-v4-pro

FP8 weights are approximately 1,600 GB. Use a persistent storage volume to avoid re-downloading on restarts. If you have multiple nodes, either mount the same NFS volume on all nodes or replicate the weights to fast local NVMe on each node before starting.

Two-Node Setup (FP8, 16x H200 SXM5)

# Node 0: start Ray head
ray start --head --port=6379

# Node 1: connect to Ray head
ray start --address='<node0-ip>:6379'

# Node 0: launch vLLM across both nodes
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 2 \
  --enable-expert-parallel \
  --quantization fp8 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

With 2-node FP8, you get ~656 GB of KV headroom after weights. This supports 128K tokens comfortably and can push to 256K with tight packing, but not 1M.

Three-Node Setup (FP8, 24x H200 SXM5, Recommended for 1M Context)

# Node 0: start Ray head
ray start --head --port=6379

# Node 1 and Node 2: connect
ray start --address='<node0-ip>:6379'

# Node 0: launch with PP=3 and full 1M context
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 3 \
  --enable-expert-parallel \
  --quantization fp8 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 1048576 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

Set NCCL environment variables before launching to enable InfiniBand RDMA:

export NCCL_IB_DISABLE=0
export NCCL_IB_HCA=mlx5_0:1
export NCCL_IB_GID_INDEX=3
export NCCL_DEBUG=WARN
export NCCL_NET_GDR_READ=1

Single-Node Setup (NVFP4, 8x B200 SXM6)

B200 SXM6 supports NVFP4 quantization in hardware, which halves the weight footprint from ~1,600 GB (FP8) to ~800 GB. An 8x B200 SXM6 node (1,536 GB total VRAM) fits V4-Pro NVFP4 weights with roughly 736 GB remaining for KV cache. No Ray cluster or pipeline parallelism needed.

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --quantization modelopt_fp4 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 524288 \
  --gpu-memory-utilization 0.92 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

NVFP4 requires vLLM v0.19+ and B200 SXM6 with appropriate driver support. If your vLLM build does not yet support --quantization modelopt_fp4, check the vLLM changelog for the minimum version with B200 NVFP4 serving enabled. On H200 or H100, use --quantization awq as the INT4 fallback, but expect higher quality regression on reasoning tasks.

SGLang Alternative

SGLang also supports DeepSeek V4 with pipeline and tensor parallelism:

python -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Pro \
  --tp 8 \
  --pp 2 \
  --quantization fp8 \
  --context-length 131072

For the full SGLang multi-node launch pattern with Ray backend, see the SGLang multi-node documentation.

Test the Endpoint

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[{"role": "user", "content": "Write a Python function to solve the N-queens problem."}],
    max_tokens=512,
)
print(response.choices[0].message.content)

Memory Planning at 1M Context

KV cache memory grows linearly with context length for V4-Pro (the same sparse attention mechanism that enables 1M context keeps active KV compute bounded, but all selected KV entries must still reside in VRAM).

The 3-node FP8 H200 cluster leaves roughly 1,784 GB after weights (~3,384 GB total minus ~1,600 GB for weights). Numbers above assume FP8 KV cache (--kv-cache-dtype fp8_e5m2) and vary with head dimension and layer count.

Set --max-model-len to your actual traffic max, not 1M by default. If real requests cap at 64K tokens, set --max-model-len 65536 and recover the KV headroom for more concurrent sessions. The KV cache optimization guide covers per-session KV budgeting in more detail.

Quantization to Cut Node Count

FP8 is the default recommendation on H200. The quality hit is marginal (under 1% on standard benchmarks) and the node count drops from 3-4 to 2-3 compared to BF16.

NVFP4 on B200 SXM6 is the path to single-node V4-Pro serving. Hardware-accelerated 4-bit inference on Blackwell produces under 2% quality regression vs FP8, which is far better than software INT4 on H200. The full dynamic expert quantization background is in the dynamic expert quantization and MoE offloading guide.

AWQ INT4 on H200 shrinks weights to ~800 GB (same as NVFP4), which fits on a single 8x H200 SXM5 node (1,128 GB total VRAM) with roughly 328 GB of headroom for KV cache. The practical problem: INT4 on H100/H200 is software-emulated, so quality regression is measurably higher than NVFP4 on B200, particularly on reasoning and coding tasks where V4-Pro's quality advantage is most important.

Throughput and Cost-Per-Token on Spheron

Pricing data from the Spheron API as of 2026-06-19. For multi-node deployments, the practical pricing unit is the 8-GPU node bundle.

Cost-per-million-token math:

Cluster $/hr / 3600 sec/hr = $/sec
$/sec / throughput tok/s = $/tok
$/tok * 1,000,000 = $/M tokens

Example for 2-node H200 spot at $53/hr and 800 tok/s:

$53 / 3600 / 800 * 1,000,000 = ~$18.40/M tokens

Note on B200 spot: B200 SXM6 spot is available at $2.71/GPU (~$22/hr for an 8-GPU node), cheaper than on-demand (~$30/hr). Spot capacity can be reclaimed at any time without notice, so use it for workloads that can tolerate interruption. For sustained production serving, on-demand is the safer choice.

Compare these figures against V4-Flash: 4x H100 SXM5 at ~$16/hr on-demand delivers 3-5x higher throughput per dollar than V4-Pro. Flash is the cost-efficient default; V4-Pro is justified when quality on complex tasks is the deciding factor.

Pricing fluctuates based on GPU availability. The prices above are based on 19 Jun 2026 and may have changed. Check current GPU pricing for live rates.

When to Use V4-Pro vs V4-Flash vs Routing

For most production deployments, the practical answer is a routing layer: run V4-Flash for 90% of calls and escalate to V4-Pro for requests that trigger a complexity classifier. Task types that reliably benefit from Pro: chain-of-thought traces over 10 steps, multi-file code generation with dependency awareness, competitive programming problems at the hardest difficulty tier, and long-document synthesis where citation accuracy matters. Short agentic calls, single-turn chat, and summarization typically show no measurable quality gain with Pro.

This routing strategy also changes the spot calculus. V4-Flash spot interruptions during 30-second agentic calls lose almost no work. V4-Pro spot interruptions during 5-minute deep reasoning runs lose significant output. Reserve on-demand or reserved capacity for the Pro endpoint if you use this pattern.

DeepSeek V4-Pro's 1.6T parameter footprint makes multi-node the only viable serving path at FP8. Spheron's H200 SXM5 nodes include InfiniBand-grade interconnect and can be provisioned as a multi-node cluster for exactly this workload.

H200 SXM5 on Spheron | B200 SXM6 on Spheron | View all GPU pricing

Deploy DeepSeek V4-Pro on Spheron →