Qwen3.8 Max GPU Requirements: VRAM and Cluster Sizing (2026)

Qwen3.8 Max GPU requirements start with a number that rules out most clusters before you even open a serving framework: 2.4 trillion total parameters, with roughly 95 billion active per token. Alibaba released the weights on 8 August 2026, and the model card says it is the first time a Qwen-Max-class model has had an open release. That makes it the second open-weight model ever to cross the 2-trillion-parameter mark, after Moonshot AI's Kimi K3. If you're planning hardware for this one, the question that matters isn't which serving flag to pass. It's whether your cluster can hold the weights at all.

Alibaba previewed Qwen3.8-Max at the World AI Conference in Shanghai on July 19, 2026, then launched it officially on August 3, 2026, built on the Qwen 3.5 architectural foundation. Five days later the open checkpoints appeared on Hugging Face: `Qwen/Qwen3.8-2.4T-A95B` in BF16 (about 4,892GB) and `Qwen/Qwen3.8-2.4T-A95B-FP8` (about 2,496GB). This guide now sizes against those measured checkpoint sizes rather than the parameter-count estimates the first version of this post used.

Its direct predecessor, Qwen3.7 Max, never shipped weights; our Qwen3.7 Max guide covers that API-only release. For the only other >2T open-weight MoE model with a real deploy guide, see Kimi K3 on GPU cloud, which is the closest sizing precedent for a cluster this large.

TL;DR: Qwen3.8 Max GPU Requirements

• Model: Qwen3.8-Max has 2.4 trillion total and 95 billion active parameters; open weights shipped 8 Aug 2026 under a custom qwen3.8-max license.
• Full FP8: the official FP8 checkpoint is 2,496GB. 16x H200 (2,256GB) and 8x B300 (2,304GB) both fall short; 24x H200 (3,384GB) or two 8-GPU B300 nodes (4,608GB) fit with KV cache room.
• Parallelism: Qwen3.8-Max has 64 attention heads, so tensor parallelism must divide 64. Run TP8 per node plus pipeline parallelism across nodes, not TP24.
• Cost: Spheron H200 is $5.52/hr on-demand per GPU as of 06 Nov 2026, so the 24-GPU FP8 cluster runs $132.48/hr.
• Quantized: a community NVFP4 checkpoint (~1,447GB) fits one 8x B300 node. Check H200 availability.

Qwen3.8-Max Architecture: 2.4T Total, 95B Active, 1M Context

Qwen3.8-Max is a sparse Mixture-of-Experts model: 2.4 trillion total parameters and about 95 billion active per forward pass (about a 4% activation ratio). Per the model card's architecture overview, it has 92 layers in a hybrid layout: 69 Gated DeltaNet linear-attention layers and 23 gated full-attention layers, each followed by an MoE block with 512 experts (10 routed plus 1 shared active per token). The full-attention layers use 64 query heads over 4 KV heads.

That hybrid layout matters for sizing. Only the 23 full-attention layers keep a growing KV cache, which by our math works out to roughly 46KB per token at FP8 KV, or about 12GB for one full 262K-token request. The 69 linear-attention layers carry a fixed-size recurrent state per request instead.

The open checkpoint and the hosted product are not the same thing. The open checkpoint is text-only, always runs in thinking mode, and has a native context of 262,144 tokens, extensible to 1,010,000. The hosted Qwen3.8-Max adds vision input, a non-thinking mode, 1M context by default, and Qwen's built-in tools. Launch coverage also lists up to 128K output tokens on the hosted model.

On agentic and coding evaluations, Alibaba's numbers put it near the top of the current field. The model card reports Terminal-Bench 2.1 at 86.6, SWE-bench Pro at 67.7, FrontierSWE at 73.5, and PaperBench at 93.0, against 88.8, 64.6, no reported score, and 90.5 for GPT-5.6 Sol (max). Launch coverage of the hosted model also cites an OSWorld-Verified score of 86.1, ahead of GPT-5.6 Sol Max (83.2), Claude Fable 5 (85.0), and Gemini 3.1 Pro (76.2), plus #4 on Frontend Code Arena (1,668 Elo) and #2 on Vision Arena (1,305 points). Those last three depend on the hosted model's vision input, which the open checkpoint does not have. Treat all of these as vendor-reported figures pending independent verification, the same caveat that applies to every frontier model's launch-day benchmark table.

API access runs $2 per million input tokens, $6 per million output tokens, and $0.25 per million cached input tokens, on the provider's published pricing checked on 08 Aug 2026, with both OpenAI- and Anthropic-compatible request formats. That API pricing gives you a real ceiling to benchmark self-hosting against once you have a cluster cost, which is exactly what the pricing section below works out.

What Changed From Qwen3.7 Max

Qwen3.8-Max builds on the Qwen 3.5 architectural line rather than iterating directly on Qwen3.7 Max's foundation, and the jump in scale is the headline change: Qwen3.7 Max's total parameter count was never officially confirmed, but third-party estimates put its flagship variant around 700B-1T total parameters. Qwen3.8-Max at 2.4T total is somewhere between 2.4x and 3.4x larger by that comparison. The practical consequence is the same one every generational jump in this MoE lineage produces: the deployment math resets. A cluster sized for Qwen3.7 Max's estimated 700B-1T footprint (4x H200 or 8x H100) is roughly a third to a half the size Qwen3.8-Max needs at FP8.

Terminal-Bench 2.1 gives a rare apples-to-apples data point across the two releases: the model card scores Qwen3.7-Max at 74.5 and Qwen3.8-Max at 86.6, a 12-point gain on agentic terminal tasks, not just a parameter-count increase.

Open Weights Release: What Shipped on 8 August 2026

Here's what the release confirmed, checked against the Hugging Face API on 20 Oct 2026:

For per-precision sizing of the smaller model, see the Qwen3.8-27B GPU and VRAM requirements.

The size row is the one worth taking seriously before you provision anything. The FP8 checkpoint came in about 96GB above the naive parameter-count calculation, because non-expert tensors, embeddings, the multi-token-prediction head, and FP8 scale tensors all add up beyond what the raw MoE math accounts for. That is a smaller gap than Kimi K3's roughly 160GB, but it is still enough to push the weights past a single 8x B300 node.

Qwen3.8 Max GPU Requirements: VRAM Table by Precision

At 2.4T total parameters, here's the naive bytes-per-parameter estimate next to what the published checkpoints actually weigh:

The 4-bit gap is the widest because quantizers usually leave attention, embeddings, and other non-expert tensors at higher precision. None of these figures include KV cache, activations, or the recurrent state of the linear-attention layers, so plan your cluster with headroom above them, not exactly to them.

FP8 Full-Weight VRAM Math (2.4T Params)

FP8 is the default production precision on Hopper and Blackwell hardware, and it's the number that decides whether your cluster is one node or three. The official FP8 checkpoint is about 2,496GB of weights. That figure rules out every single 8-GPU node: 8x H100 (640GB) and 8x H200 (1,128GB) aren't close, and 8x B300 (2,304GB) is 192GB short before you allocate a single byte of KV cache. There is no flag that makes that work.

For GPU memory sizing fundamentals across model families, including how bytes-per-parameter math generalizes beyond this one model, see the general VRAM sizing guide.

Why Active Params (95B) Don't Shrink Your GPU Bill

The 95B active-parameter figure describes compute per token, not memory footprint, and conflating the two is the single most common MoE sizing mistake. Qwen3.8-Max routes each token through a subset of its expert pool, but the router can send any token to any expert on any given forward pass. That means every expert's weights have to be resident in VRAM at all times, whether or not they activate on this particular token. Active parameters tell you how fast inference runs. Total parameters tell you how many GPUs you need. Size your cluster off the 2.4T figure, full stop.

This is the same distinction DeepSeek V4 and Kimi K3 deployments run into: as our DeepSeek V4 deployment guide covers, V4-Pro's 49B active parameters out of 1.6T total don't shrink its ~865GB native checkpoint one byte, and Kimi K3's 104B active out of 2.8T total doesn't change its 1.56TB real checkpoint size. For a deeper look at how expert routing and parallelism interact with this constraint, see the MoE fine-tuning and expert parallelism guide.

GPU Cluster Sizing: What It Takes to Serve the Full Weights vs a Quantized Cut

The short version: full FP8 needs a real multi-node cluster, not a single 8-GPU box, and the GPU count also has to map onto a legal parallel layout. Qwen3.8-Max has 64 attention heads, so the tensor-parallel size must be 1, 2, 4, 8, 16, 32, or 64. Anything else (TP24, TP12, TP20) fails at startup in vLLM and SGLang. For GPU counts that aren't a power of two, run TP8 inside each node and pipeline parallelism across nodes.

Full FP8 Cluster: 24x H200 or 2x B300 Nodes

On H200 SXM5 (141GB per GPU), the arithmetic against the 2,496GB FP8 checkpoint breaks down like this. The last column assumes --gpu-memory-utilization 0.9, which reserves 10% of each GPU for the runtime:

Sixteen H200s, the tidy two-node number, can't hold the weights. Twenty H200s technically can, but after the runtime reserve there's about 42GB left across the whole cluster for KV cache, activations, CUDA graphs, and linear-attention state, which is a few long requests at most, and 20 GPUs is two and a half nodes anyway. A 3-node, 24-GPU H200 cluster run as TP8 x PP3 is the practical target for a production FP8 deployment. The vLLM recipe for Qwen3.8 sizes FP8 at 32 H200s because it plans for pure tensor parallelism and more KV room; go to four nodes if you need long contexts at high concurrency.

B300 SXM6 tells a simpler story than this post's first version suggested. A single 8-GPU B300 node totals 2,304GB, 192GB short of the FP8 checkpoint, so it cannot serve FP8 at all; use it for the 4-bit checkpoints below. Two 8-GPU B300 nodes (4,608GB) leave about 2,100GB of raw headroom and run as TP16 across both nodes, which is the FP8 reference layout in the vLLM recipe, or TP8 x PP2.

B200 SXM6 (192GB per GPU) also needs two nodes: a single 8-GPU B200 node totals just 1,536GB, well short of the FP8 requirement. Sixteen B200s across two nodes gives 3,072GB, about 576GB of raw headroom, runs as TP16 or TP8 x PP2, and mirrors the exact topology Kimi K3 uses for its own two-node B200 MXFP4 deployment.

Quantized Cuts: INT4/MXFP4 on Smaller Clusters

Alibaba has not published an official 4-bit checkpoint, but community conversions appeared within days of the release. The most-used are Inferact's NVFP4 W4A4 checkpoint at about 1,447GB and AMD's Quark MXFP4 checkpoint at about 1,373GB. Here's how they fit:

A single 8-GPU B300 node is the cleanest quantized entry point: it holds the NVFP4 checkpoint with real margin for KV cache, and it's one node instead of a multi-node launch. A single B200 node loads it, but with under 90GB spare across eight GPUs you'll be capping --max-model-len and concurrency hard. Quality on 4-bit conversions depends on calibration, and these are community checkpoints rather than Alibaba's own, so run your eval set against the FP8 model before you commit to one.

How Qwen3.8-Max Cluster Sizing Compares to Kimi K3 and DeepSeek V4

Lining up the three large open-weight MoE models by total parameter count makes the sizing gap concrete:

Kimi K3's checkpoint is MXFP4 natively, which is why its footprint (1.56TB) is smaller than Qwen3.8-Max's official FP8 checkpoint (2.5TB) despite having more total parameters. That's a precision comparison, not a parameter-count one: Qwen3.8-Max's community NVFP4 checkpoint at ~1.45TB lands in the same range as Kimi K3 and fits the same single 8x B300 node. At official precision, though, Qwen3.8-Max is the only one of the three that needs more than one B300 node.

Deploy Qwen3.8-Max on GPU Cloud with vLLM and SGLang

Both frameworks had day-0 support, the same pattern our Qwen 3.6 guide documents for the open Qwen3.6-35B-A3B and Qwen3.6-27B models. vLLM supports the Qwen3_5MoeForCausalLM architecture in its nightly builds per the vLLM Qwen3.8 recipe, and SGLang ships a launch image, lmsysorg/sglang:qwen38, covered in the SGLang Qwen3.8 cookbook. Check both pages for flag changes before you launch, since support for a brand-new architecture keeps moving for weeks after release.

Prerequisites and Weight Download

• A Spheron account with access to your target GPU configuration (app.spheron.ai)
• A Hugging Face account and read-scoped access token
• At least 3TB of free local NVMe per node for the 2,496GB FP8 checkpoint, or a shared volume all nodes can mount
• vLLM nightly, or Docker with the NVIDIA container toolkit on each node for the SGLang image

Download the official FP8 checkpoint:

python3 -m pip install -q huggingface_hub hf_transfer
export HF_TOKEN="hf_your_token_here"

nohup env HF_HUB_ENABLE_HF_TRANSFER=1 HF_TOKEN="$HF_TOKEN" \
  python3 -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3.8-2.4T-A95B-FP8', local_dir='/data/models/qwen38max-fp8', max_workers=16)" \
  > /tmp/dl.log 2>&1 &

HF_HUB_ENABLE_HF_TRANSFER=1 matters at this size: a 2.5TB download at unaccelerated throughput can take hours, while accelerated transfer typically finishes in under an hour on a multi-gigabit connection, the same speedup a 1.56TB Kimi K3 checkpoint sees under the identical flag.

Multi-Node Launch Commands (Tensor + Pipeline Parallelism)

For a 3-node, 24-GPU H200 FP8 deployment with vLLM, use TP8 inside each node and three pipeline stages across nodes. Run the same command on every node, changing only --node-rank; ranks 1 and 2 also take --headless so they don't start a second API server:

vllm serve /data/models/qwen38max-fp8 \
    --served-model-name Qwen/Qwen3.8-Max \
    --tensor-parallel-size 8 \
    --pipeline-parallel-size 3 \
    --nnodes 3 \
    --node-rank 0 \
    --master-addr $HEAD_ADDR \
    --kv-cache-dtype fp8 \
    --gpu-memory-utilization 0.9 \
    --max-model-len 65536 \
    --reasoning-parser qwen3 \
    --host 0.0.0.0 \
    --port 8000

No --quantization flag is needed: the FP8 checkpoint carries its quantization config. For a single 8-GPU B300 node, switch to the NVFP4 checkpoint, since FP8 won't fit:

vllm serve Inferact/Qwen3.8-2.4T-A95B-NVFP4 \
    --served-model-name Qwen/Qwen3.8-Max \
    --tensor-parallel-size 8 \
    --kv-cache-dtype fp8 \
    --gpu-memory-utilization 0.9 \
    --max-model-len 65536 \
    --reasoning-parser qwen3 \
    --host 0.0.0.0 \
    --port 8000

The equivalent SGLang launch for FP8 across two 8-GPU B300 nodes, with TP16 (a legal divisor of 64):

python -m sglang.launch_server \
    --model-path /data/models/qwen38max-fp8 \
    --served-model-name Qwen/Qwen3.8-Max \
    --tp 16 \
    --nnodes 2 \
    --node-rank $RANK \
    --dist-init-addr $HEAD_IP:20000 \
    --reasoning-parser qwen3 \
    --host 0.0.0.0 \
    --port 30000

On a 3-node H200 cluster, the SGLang equivalent is --tp 8 --pp-size 3 --nnodes 3. Never pass --tp 24: 24 does not divide 64 attention heads.

The architecture is native in both frameworks, so --trust-remote-code is no longer needed. Set --max-model-len to your real workload: the vLLM recipe found that the same KV pool that held 25 concurrent requests at 262,144 tokens served 506 at about 9K tokens. To go past the native 262K window toward 1,010,000 tokens, follow the long-context instructions in the vLLM recipe, which override max_position_embeddings.

Validating the Deployment

Once the server is up, confirm it's serving correctly before routing production traffic:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen/Qwen3.8-Max",
        "messages": [{"role": "user", "content": "Explain sparse MoE routing in three sentences."}],
        "max_tokens": 512
    }'

The open checkpoint always thinks first, so a 512-token cap can end inside the reasoning block; raise max_tokens if the answer comes back empty. Then check VRAM usage matches expectations with nvidia-smi across every GPU in the cluster, not just the head node. On a 24-GPU H200 deployment, each GPU should show roughly 104GB of resident weights plus KV cache; materially less on any single GPU usually means a rank failed to load its shard. Watch throughput with nvidia-smi dmon -s pum -d 5 and check the framework's metrics endpoint (/metrics on vLLM) for TTFT, ITL, and KV cache saturation once real traffic starts flowing.

Qwen3.8 Max vs Kimi K3 vs DeepSeek V4: Size, Cost, Benchmarks

Putting size, hardware, and reported benchmarks side by side:

DeepSeek V4-Pro gets its million-token context from a hybrid of Compressed Sparse Attention and Heavily Compressed Attention, not the DSA design V3.2 used. Kimi K3 remains the largest open-weight model by parameter count. Qwen3.8-Max is the second model in that >2T tier and the largest of the three on disk at official precision, and it is the first time Alibaba has open-sourced a Qwen-Max-class flagship at all, a real departure from its prior practice of keeping the Max tier API-only. DeepSeek V4 remains the accessible entry point for teams that don't need the extra scale: V4-Pro serves from a single 8x H200 node, and the smaller V4-Flash (284B total/13B active, ~160GB) fits on 2x H200, versus a multi-node cluster for Qwen3.8-Max at FP8.

If your workload doesn't strictly need a 2T+ parameter model, DeepSeek V4's smaller footprint is worth weighing against the operational complexity of standing up a multi-node MoE cluster. If it does, budget for the larger cluster from day one rather than discovering mid-provisioning that your two-node H200 cluster is 240GB short of fitting the weights.

Spheron GPU Pricing for Qwen3.8-Max Workloads

Spheron rates below are live. Multi-GPU cluster rates are calculated as the lowest available per-GPU rate times GPU count, an approximation since actual bundle offers vary by region and availability.

At current API pricing of $2/M input and $6/M output tokens, a 24x H200 on-demand cluster at $132.48/hr costs more per month than the API bill for anything short of billions of tokens per month (the exact point depends on your input/output mix); run the exact math against your own traffic before committing to self-hosting at this scale. Self-hosting only pays off once volume is high enough to amortize a genuinely large fixed cost, and a cluster this size is not the place to guess.

Pricing fluctuates based on GPU availability. Spheron node totals above are live as of 06 Nov 2026; the YottaLabs API rate is their published figure, checked 08 Aug 2026. Check current GPU pricing → for live rates.

Troubleshooting Multi-Node Qwen3.8-Max Deployments

OOM at roughly 90%+ of weight loading: The clearest sign your cluster is undersized relative to the real checkpoint. This is exactly what happened when 8x B200 (1,536GB) tried to load Kimi K3's real 1,560.94GB checkpoint, and it's what a single 8x B300 node (2,304GB) does with Qwen3.8-Max's 2,496GB FP8 checkpoint. Move to the next tier up, or to a 4-bit checkpoint, rather than fighting --gpu-memory-utilization down further.

Process group hangs at multi-node startup: Verify every node has a distinct rank (0 through N-1, no duplicates) and that the full TCP port range NCCL and Gloo need is open between nodes. A partial firewall rule causes a hang at the barrier rather than a clear error.

NCCL timeout across nodes: Set NCCL_TIMEOUT=1800 and NCCL_DEBUG=INFO to diagnose. NVLink-connected SXM5/SXM6 nodes avoid most of this; PCIe-connected multi-node setups are slower to negotiate and more prone to timing out on the default window. Running nccl-tests to benchmark the cluster's interconnect bandwidth before you launch this deployment catches a degraded fabric early, instead of debugging a hang mid-load.

Model class not recognized: Qwen3_5MoeForCausalLM needs a current build: vLLM nightly, or SGLang's lmsysorg/sglang:qwen38 image. An older stable release fails here, and --trust-remote-code won't fix it.

Tensor-parallel size rejected: The tensor-parallel size has to divide Qwen3.8-Max's 64 attention heads, so 1, 2, 4, 8, 16, 32, and 64 are the only legal values. On 24 GPUs, run --tensor-parallel-size 8 --pipeline-parallel-size 3, not TP24. Pipeline parallelism is slower than pure TP or expert parallelism, so prefer 16 or 32 GPUs when you have the choice.

CUDA graph or recurrent-state cache assertion at startup: On this hybrid architecture, an num_cache_lines >= batch failure means the CUDA-graph capture size exceeds the linear-attention state cache. The vLLM recipe's fix is to lower --max-cudagraph-capture-size from its default of 512.

Suspiciously tight KV cache headroom after weights load: If you provisioned close to the checkpoint size (20x H200, or one B200 node on NVFP4) and see almost no room left for KV cache, that's the headroom problem, not a misconfiguration. The SGLang cookbook also notes that the linear-attention state, not the KV cache, is often what caps concurrency on this model. Move to the next cluster tier rather than trying to reclaim memory with lower --max-model-len alone.

Qwen3.8-Max's official FP8 checkpoint is 2,496GB, which rules out every single 8-GPU node, 8x B300 included. Size for a 24x H200 or two-node B300 cluster, or drop to a 4-bit checkpoint on one B300 node, and leave headroom.

H200 SXM5 on Spheron → | Spheron B300 instances → | View all GPU pricing →