Deploy DeepSeek V4 on GPU Cloud: MoE Inference with vLLM and Expert Parallelism

Last updated: September 30, 2026

DeepSeek V4 is out as two open-weight Mixture-of-Experts models under the MIT license, both with a 1M-token context window: V4-Pro, with 1.6T total parameters and 49B active per token, and V4-Flash, with 284B total and 13B active. This guide covers the deployment path for both: sizing VRAM from the real checkpoint, picking a GPU configuration, running vLLM with expert parallelism, and tuning for production throughput. DeepSeek has since shipped a larger successor with image input; see the DeepSeek V4.1 Flash GPU requirements guide for the 552B model.

This post is the overview for the whole V4 family. For variant-specific tuning, the DeepSeek V4-Flash deployment guide goes deeper on the 284B model, and the DeepSeek V4-Pro deployment guide covers the 1.6T model, including multi-node setups for long-context workloads.

For a DeepSeek V4 vs GLM-5.1 vs GPT-OSS 120B benchmark comparison, see the open-weight frontier model showdown post. If you are also considering GLM-5.1 as an alternative, the GLM-5.1 deployment guide covers a similar 8x H200 FP8 setup for a model with slightly higher Chatbot Arena scores on coding tasks. If your workload involves document processing, the DeepSeek family also includes DeepSeek-OCR, which runs efficiently on the same GPUs for PDF and image extraction. If your primary constraint is 1M+ token context at affordable TTFT, SubQ 1M-Preview with 12M context trades MoE model quality for linear prefill scaling and fits the full context window on a single H200 node. For a smaller multi-GPU footprint at comparable active-parameter scale, see the Mistral Large 3 deployment guide. For context on the Huawei Ascend hardware that was part of DeepSeek's training compute, the Ascend 950 vs B200 and B300 inference comparison explains why most teams outside China use NVIDIA for DeepSeek inference serving.

TL;DR: Deploy DeepSeek V4 on GPU Cloud

• Models: DeepSeek V4-Pro is 1.6T total / 49B active; V4-Flash is 284B total / 13B active. Both have a 1M-token context and MIT-licensed weights.
• V4-Pro VRAM: the native FP4+FP8 checkpoint is about 865GB, so it fits one 8x H200 141GB node (1,128GB) or 8x B200, but not 8x H100 80GB (640GB).
• V4-Flash VRAM: the checkpoint is about 160GB, so it fits 2x H200 (282GB) or 4x H100; 2x H100 is too tight.
• Spheron cost: 8x H200 runs $44.16/hr on-demand and 2x H200 runs $11.04/hr, as of 15 Nov 2026, on Spheron H200 instances.

What Is DeepSeek V4

DeepSeek V4 is a family of two sparse MoE models:

• V4-Pro: 1.6T total parameters, 49B active per token. The flagship, aimed at coding, agentic and long-context reasoning work.
• V4-Flash: 284B total parameters, 13B active per token. The same design at a size that fits on two H200s.

Both use top-K routing to pick which experts process each token, which keeps per-token compute tied to the active count rather than the total. Both ship with a 1M-token context window and open weights under the MIT license.

Compared to DeepSeek V3 (671B total, 37B active, 128K context), V4-Pro is bigger on both counts: more than twice the total parameters and 49B active instead of 37B, so you pay more for storage, weight loading and per-token compute. V4-Flash goes the other way, with fewer total and active parameters than V3, which makes it the cheaper model to host of the three. Both V4 models raise the context window from 128K to 1M tokens.

For background on why active parameter count doesn't reduce memory requirements, see our GPU memory requirements guide for LLMs.

GPU Hardware Requirements for DeepSeek V4

DeepSeek publishes V4 as a mixed-precision checkpoint: FP4 for the MoE expert weights and FP8 for everything else. That brings V4-Pro to about 865GB on disk and V4-Flash to about 160GB. Size your cluster from those figures plus KV cache and activation memory. For live per-precision estimates and the cheapest matching GPUs, see the DeepSeek V4 VRAM requirements breakdown.

VRAM math: do not size V4 as parameters times one byte. The native checkpoint stores expert weights in FP4, which is why V4-Pro is ~865GB rather than 1.6TB. Converting to pure FP8 roughly doubles the expert weights, so V4-Pro at FP8 is about 1.6TB and no longer fits a single 8x H200 node. Re-quantizing to 4-bit does not help either: real INT4 and GGUF Q4_K_M files land at about 0.6 bytes per parameter, roughly 960GB for V4-Pro and 170GB for V4-Flash, which is larger than the native checkpoint.

The active parameter count (49B for Pro, 13B for Flash) does not reduce VRAM. All expert weights must be loaded into GPU memory because the router selects different experts per token. You can't lazy-load experts without major latency spikes.

For hardware comparisons between H100 and H200, see H100 vs H200: which GPU for LLM inference. For a quick lookup of memory requirements across different models, see our GPU requirements cheat sheet 2026.

Step-by-Step Deployment with vLLM

Prerequisites

• A recent vLLM release with DeepSeek V4 support (check the vLLM release notes for the minimum version)
• CUDA 12.4+, Python 3.10+
• Hugging Face account to download deepseek-ai/DeepSeek-V4-Pro or deepseek-ai/DeepSeek-V4-Flash
• 8x H200 SXM5 or 8x B200 for V4-Pro; 2x H200 SXM5 or 4x H100 SXM5 for V4-Flash

Install vLLM

pip install vllm  # installs latest; confirm the release notes list DeepSeek V4 support

Download Model Weights

# V4-Pro: ~865GB
huggingface-cli download deepseek-ai/DeepSeek-V4-Pro --local-dir ./deepseek-v4-pro

# V4-Flash: ~160GB
huggingface-cli download deepseek-ai/DeepSeek-V4-Flash --local-dir ./deepseek-v4-flash

The V4-Pro download is about 865GB and V4-Flash about 160GB. Use a persistent storage volume so you don't re-download on instance restarts.

Launch V4-Pro on 8x H200 (Tensor Parallelism)

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --host 0.0.0.0 \
  --port 8000

This splits every transformer layer across all 8 GPUs simultaneously. All 8 GPUs participate in every forward pass, which minimizes time-to-first-token at the cost of higher NVLink bandwidth usage. vLLM picks up the checkpoint's FP4+FP8 quantization from the model config, so there is no precision flag to set.

For V4-Flash, use the same command with deepseek-ai/DeepSeek-V4-Flash and --tensor-parallel-size 2 on 2x H200, or --tensor-parallel-size 4 on 4x H100.

Launch with Mixed Expert and Tensor Parallelism

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --host 0.0.0.0 \
  --port 8000

--tensor-parallel-size splits attention layers across GPUs. --enable-expert-parallel activates expert parallelism, splitting the MoE expert layers across GPUs so each GPU holds a subset of whole experts and the router dispatches tokens to the GPU that holds the relevant expert. The EP size is auto-calculated from TP and DP sizes when this flag is enabled.

For DeepSeek V4's performance ceiling, the DeepEP and DeepGEMM deployment guide covers the kernel-level setup that takes expert parallelism from baseline NCCL to overlapped NVSHMEM dispatch.

When to use each:

• Pure tensor parallelism: better for latency-sensitive serving; every expert is sharded across all GPUs, so all GPUs are always active
• Mixed TP+EP: better for throughput-maximizing workloads where you can tolerate higher TTFT; whole experts live on one GPU, which trades the per-layer all-reduce on expert weights for all-to-all token dispatch

DeepSeek V4's native sparse attention architecture can be activated via the DSA backend: see the DSA inference guide for step-by-step configuration of --attention-backend dsa alongside expert parallelism.

Test the Endpoint

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",  # or deepseek-ai/DeepSeek-V4-Flash
    messages=[{"role": "user", "content": "Write a Python function to binary search a sorted list."}],
    max_tokens=512,
)
print(response.choices[0].message.content)

For full vLLM setup details including Docker deployment, monitoring, and load balancing, see the vLLM production deployment guide. For setting up an OpenAI-compatible API layer for self-hosted models, see the self-hosted OpenAI-compatible API guide.

Hardware Comparison: H200 vs B200 vs H100 vs A100

Throughput depends heavily on batch size, context length and parallelism layout, so benchmark your own traffic before committing. What decides the shortlist is whether the weights fit with room left for KV cache:

For V4-Pro, the choice is between 8x H200 and 8x B200. B200 adds headroom for long contexts and more concurrent sessions, and its Blackwell tensor cores handle FP4 natively, while H200 and H100 have no FP4 tensor cores. For V4-Flash, 2x H200 is the smallest footprint and 4x H100 is the alternative when H200 capacity is short. A100 lacks the Hopper Transformer Engine used for hardware-accelerated FP8, so it only makes sense if you already own the capacity.

For a detailed comparison of vLLM, TensorRT-LLM, and SGLang throughput numbers, see vLLM vs TensorRT-LLM vs SGLang benchmarks.

Spheron GPU Pricing for DeepSeek V4

Spheron rates for the configurations above, resolved from the live pricing API:

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 15 Nov 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

Spot instances are viable for batch inference jobs: document processing pipelines, offline evaluation, or any workload where you can handle interruptions, since spot capacity can be reclaimed without notice. For real-time serving with SLA requirements, on-demand is the right choice.

Whether self-hosting beats a per-token API depends on your monthly token volume and context length distribution. For the full break-even model against DeepSeek's own endpoint, including data-privacy considerations, see the DeepSeek API pricing vs self-hosted comparison.

A third route sits between the two: Spheron's fixed-price inference plan serves DeepSeek V4.1 Flash, the successor to V4-Flash, through an OpenAI-compatible API at a monthly price fixed for the term, with no GPUs to run. Teams that need V4-Pro itself can ask about a Dedicated plan, which runs another open model on single-tenant nodes pinned to the EU or US.

For a full competitive pricing comparison across GPU cloud providers, see GPU cloud pricing comparison 2026.

Production Optimization

Precision: Stay on the Native Checkpoint

The native FP4+FP8 checkpoint is the right default for DeepSeek V4. It is the format DeepSeek released, so there is no conversion step and no extra quantization loss to test for. The alternatives all cost memory:

• Pure FP8: roughly doubles the expert weights. V4-Pro grows to about 1.6TB, which no longer fits one 8x H200 node.
• BF16: 2 bytes per parameter, about 3.2TB for V4-Pro and 570GB for V4-Flash. Only worth it for research that needs unquantized weights.
• INT4 / GGUF Q4_K_M: about 0.6 bytes per parameter, roughly 960GB for V4-Pro and 170GB for V4-Flash. Both are larger than the native checkpoint, so re-quantizing saves nothing.

Quantization decision guide:

• Native FP4+FP8 (default): V4-Pro on 8x H200 or 8x B200, V4-Flash on 2x H200 or 4x H100
• FP8 or BF16: only when a toolchain can't read the mixed-precision checkpoint, and only with a multi-node budget for V4-Pro
• 4-bit re-quantization: skip it for V4; the native checkpoint is already smaller

KV Cache Tuning

FP8 KV cache cuts memory usage by half compared to BF16 KV cache:

vllm serve deepseek-ai/DeepSeek-V4-Pro \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_e5m2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90

--max-model-len vs --gpu-memory-utilization tradeoff:

For most applications, 32768 tokens is a practical ceiling. Both V4 models accept up to 1M tokens, but very few real requests use anything close to that, and reserving VRAM for a maximum context length you never use leaves fewer slots for concurrent users. On 8x H200 the ~260GB left after V4-Pro's weights is the budget every session shares.

For a deeper dive on PagedAttention, prefix caching, and CPU offloading, see the KV cache optimization guide.

Continuous Batching and Throughput

Enable chunked prefill for long-context requests:

--enable-chunked-prefill

Tune maximum concurrent sequences for your latency target:

--max-num-seqs 32   # lower = lower latency, higher = higher throughput

Monitor GPU utilization during load testing:

nvidia-smi dmon -s u -d 5

Watch for GPU utilization below 70% on loaded instances: that usually means --max-num-seqs is too low and the GPU is waiting for more concurrent requests.

For additional throughput techniques including speculative decoding, see the speculative decoding production guide. For an additional 2-4x throughput gain on top of base expert-parallel serving, see draft-head speculative decoding for DeepSeek V4. For block-diffusion variants that pair well with vLLM on Blackwell GPUs, see our dFlash block-diffusion speculative decoding guide.

DeepSeek V4 vs V3 vs Llama 4 Maverick

\*R2 figures are unconfirmed; see the R2 deployment guide linked below for the latest details.

Decision guidance:

Choose DeepSeek V4-Pro when coding quality and agentic reasoning are the primary requirements and you can dedicate a full 8x H200 or 8x B200 node to it.

Choose DeepSeek V4-Flash when you want a 1M-context DeepSeek model on a small footprint: 2x H200 or 4x H100, with 13B active parameters keeping per-token compute low.

Choose DeepSeek R2 when reasoning accuracy on math, logic, or science tasks is the priority over coding throughput. See the DeepSeek R2 deployment guide for hardware requirements and vLLM configuration specific to reasoning workloads.

Choose DeepSeek V3 if you want a more established model with a larger community of deployment guides, known quirks, and benchmarked configurations. V3 is production-stable; V4 is newer.

Choose Llama 4 Maverick when you prefer Meta's open-weights license or want a middle point between the two V4 models (17B active, against 49B for V4-Pro and 13B for V4-Flash).

For Llama 4 deployment steps, see Deploy Llama 4 on GPU cloud. For GPU hardware selection guidance, see Best GPU for AI inference 2026. Teams evaluating Alibaba's alternatives should note that Qwen3.7 Max is API-only, so it can be compared on quality through its API but not self-hosted on the same hardware.

DeepSeek V4-Pro needs a full 8x H200 or B200 node and V4-Flash runs on two H200s, and Spheron has both configurations on demand without you managing physical hardware. Spot capacity is available for batch workloads that can handle interruptions.

H100 SXM5 on Spheron → | Spheron B200 instances → | View all GPU pricing →

Deploy DeepSeek V4 on Spheron →