Qwen3.8 Max GPU Requirements: VRAM and Cluster Sizing (2026)

Qwen3.8 Max GPU requirements start with a number that rules out most clusters before you even open a serving framework: 2.4 trillion total parameters, with roughly 95 billion active per token. Alibaba has committed to open-sourcing it, the first time it has ever open-sourced a Qwen-Max-class model (DataCamp, YottaLabs). That makes it the second open-weight model ever to cross the 2-trillion-parameter mark, after Moonshot AI's Kimi K3. If you're planning hardware for this one, the question that matters isn't which serving flag to pass. It's whether your cluster can hold the weights at all.

Alibaba previewed Qwen3.8-Max at the World AI Conference in Shanghai on July 19, 2026, then launched it officially on August 3, 2026, built on the Qwen 3.5 architectural foundation (TestingCatalog). Open weights for both Qwen3.8-Max and a smaller dense sibling, Qwen3.8-27B, are confirmed for the week of August 10, 2026, on Hugging Face and ModelScope (TestingCatalog). As of this writing, no matching repository has appeared yet on the Qwen Hugging Face organization page, so this guide works from the announced architecture, not a measured checkpoint. We flag every estimate as such and update the hard numbers once the weights land.

For the direct predecessor and what changed architecturally, see the Qwen3.7 Max deployment guide. For the only other confirmed >2T open-weight MoE model with a real deploy guide, see Kimi K3 on GPU cloud, which is the closest sizing precedent for a cluster this large.

Qwen3.8-Max Architecture: 2.4T Total, 95B Active, 1M Context

Qwen3.8-Max is a sparse Mixture-of-Experts model: 2.4 trillion total parameters, approximately 95 billion active per forward pass (about a 4% activation ratio), a 1M-token context window, and up to 128K output tokens (DataCamp). It supports native text, image, and video input.

On agentic and coding evaluations, Alibaba's reported numbers put it near the top of the current field. OSWorld-Verified, a benchmark for autonomous desktop computer-use tasks, scores 86.1, ahead of GPT-5.6 Sol Max (83.2), Claude Fable 5 (85.0), and Gemini 3.1 Pro (76.2). PaperBench comes in at 93.0 (DataCamp). SWE-bench solve rate is reported at 87.3%, Terminal-Bench 2.1 at 67.4, with a #4 overall placement on Frontend Code Arena (1,668 Elo) and #2 on Vision Arena (1,305 points) (Latent.Space). Treat these as vendor-reported figures pending independent verification, the same caveat that applies to every frontier model's launch-day benchmark table.

API access is live today at $2 per million input tokens, $6 per million output tokens, and $0.25 per million cached input tokens, with both OpenAI- and Anthropic-compatible request formats (YottaLabs). That API pricing gives you a real ceiling to benchmark self-hosting against once you have a cluster cost, which is exactly what the pricing section below works out.

What Changed From Qwen3.7 Max

Qwen3.8-Max builds on the Qwen 3.5 architectural line rather than iterating directly on Qwen3.7 Max's foundation, and the jump in scale is the headline change: Qwen3.7 Max's total parameter count was never officially confirmed, but third-party estimates put its flagship variant around 700B-1T total parameters. Qwen3.8-Max at 2.4T total is somewhere between 2.4x and 3.4x larger by that comparison. The practical consequence is the same one every generational jump in this MoE lineage produces: the deployment math resets. A cluster sized for Qwen3.7 Max's estimated 700B-1T footprint (4x H200 or 8x H100) is roughly a third to a half the size Qwen3.8-Max needs at FP8.

Terminal-Bench 2.1 gives a rare apples-to-apples data point across the two releases: Qwen3.7 Max scored 61.0, Qwen3.8-Max scores 67.4 (Latent.Space), a real generational improvement on agentic terminal tasks, not just a parameter-count increase.

Open Weights Timeline: What's Confirmed vs Still Pending

Here's what's locked in and what isn't, as of this post's publication date:

That last row is the one worth taking seriously before you provision anything. Kimi K3's published checkpoint came in about 160GB larger than the naive parameter-count calculation predicted, because non-expert tensors, embeddings, and activation scale blocks all add up beyond what the raw MoE math accounts for. Expect a similar gap here. We build the estimates below on the naive calculation and flag where real-world overhead is likely to bite.

Qwen3.8 Max GPU Requirements: VRAM Table by Precision

At 2.4T total parameters, here's what weight storage looks like at each common precision, using the standard bytes-per-parameter convention:

These are theoretical minimums from parameter count alone, before KV cache, activations, or the real-world overhead that consistently pushes published checkpoints above the naive figure. Plan your cluster with headroom above these numbers, not exactly to them.

FP8 Full-Weight VRAM Math (2.4T Params)

FP8 is the default production precision on Hopper and Blackwell hardware, and it's the number that decides whether your cluster is one node or three. 2.4T parameters at 1 byte each gives roughly 2,400GB of raw weight storage. That figure alone rules out any single 8-GPU H100 or H200 node: 8x H100 (640GB) and 8x H200 (1,128GB) aren't close, and even 8x B300 (2,304GB) sits right at the edge with essentially no margin for KV cache, activations, or the checkpoint-size overhead the Kimi K3 precedent suggests is coming.

For GPU memory sizing fundamentals across model families, including how bytes-per-parameter math generalizes beyond this one model, see the general VRAM sizing guide.

Why Active Params (95B) Don't Shrink Your GPU Bill

The 95B active-parameter figure describes compute per token, not memory footprint, and conflating the two is the single most common MoE sizing mistake. Qwen3.8-Max routes each token through a subset of its expert pool, but the router can send any token to any expert on any given forward pass. That means every expert's weights have to be resident in VRAM at all times, whether or not they activate on this particular token. Active parameters tell you how fast inference runs. Total parameters tell you how many GPUs you need. Size your cluster off the 2.4T figure, full stop.

This is the same distinction DeepSeek V4 and Kimi K3 deployments run into: DeepSeek V4's 37B active parameters out of 1T total don't shrink its ~500GB FP8 footprint one byte (DeepSeek V4 deployment guide), and Kimi K3's 104B active out of 2.8T total doesn't change its 1.56TB real checkpoint size. For a deeper look at how expert routing and parallelism interact with this constraint, see the MoE fine-tuning and expert parallelism guide.

GPU Cluster Sizing: What It Takes to Serve the Full Weights vs a Quantized Cut

The short version: full FP8 needs a real multi-node cluster, not a single 8-GPU box, and the smallest configuration that looks like it fits on paper is usually the one to be most suspicious of.

Full FP8 Cluster: 16x-20x H200 or 8x B300

On H200 SXM5 (141GB per GPU), the arithmetic against a 2,400GB naive FP8 estimate breaks down like this:

Sixteen H200s, the tidy two-node number, comes up short even before you account for KV cache or the checkpoint-overhead margin that real MoE releases tend to carry. A 3-node, 24-GPU H200 cluster is the practical target for a production FP8 deployment: it clears the naive weight estimate with real room for KV cache, activations, and whatever overhead the actual checkpoint adds once Alibaba publishes it.

B300 SXM6 tells a more interesting story. A single 8-GPU B300 node totals 2,304GB, which is genuinely close to the 2,400GB naive FP8 estimate, close enough that it's within the same margin of error that made an 8x B200 node fall 25GB short of Kimi K3's real 1.56TB checkpoint. Treat single-node 8x B300 as the floor for Qwen3.8-Max, not a comfortable production target, until the real checkpoint size is public. Two 8-GPU B300 nodes (4,608GB) removes that risk entirely and is the safer call for anything beyond a first evaluation run.

B200 SXM6 (192GB per GPU) needs at least two nodes: a single 8-GPU B200 node totals just 1,536GB, well short of the FP8 requirement. Sixteen B200s across two nodes gives 3,072GB, comfortable relative to the naive estimate, and mirrors the exact topology Kimi K3 uses for its own two-node B200 MXFP4 deployment.

Quantized Cuts: INT4/MXFP4 on Smaller Clusters

At INT4 or MXFP4 (0.5 bytes/param), the naive weight estimate drops to roughly 1,200GB, which opens up meaningfully smaller clusters:

A single 8-GPU B200 node is the cleanest quantized entry point: it fits the ~1,200GB estimate with real margin left for KV cache, and it's a topology you can provision as one node instead of coordinating a multi-node launch. The 16x H100 configuration works but leaves less room, and quality on INT4/MXFP4 checkpoints depends heavily on calibration quality once community or official quantized weights actually exist. Neither Alibaba's official MXFP4/INT4 checkpoint nor community AWQ/GPTQ conversions have been published as of this writing, so treat this section as a sizing plan for when they do, not a deployment you can run today.

How Qwen3.8-Max Cluster Sizing Compares to Kimi K3 and DeepSeek V4

Lining up the three confirmed large open-weight MoE models by total parameter count makes the sizing gap concrete:

Kimi K3's real checkpoint is MXFP4 natively, which is why its confirmed footprint (1.56TB) is smaller than Qwen3.8-Max's FP8 estimate (2.4TB) despite having more total parameters. That's a precision comparison, not a parameter-count one: Qwen3.8-Max at MXFP4 would land closer to 1.2TB, in the same range as Kimi K3's real number. The honest read is that Qwen3.8-Max's practical cluster size, once real weights land, will probably sit somewhere between DeepSeek V4's single-node footprint and Kimi K3's multi-node one, and closer to Kimi K3's given the total parameter count involved.

Deploy Qwen3.8-Max on GPU Cloud with vLLM and SGLang

Qwen releases have historically shipped with day-one support in both vLLM and SGLang, the same pattern the site's Qwen 3.6 Plus deployment guide documents for the prior generation, and there's no reason to expect Qwen3.8-Max to break that pattern. Both frameworks are the expected path once weights land. The commands below are the standard MoE launch pattern for a model this size; verify exact flag names against each framework's release notes once Qwen3.8-Max support ships, since flag names for brand-new architectures sometimes shift between nightly and stable builds.

Prerequisites and Weight Download

• A Spheron account with access to your target GPU configuration (app.spheron.ai)
• A Hugging Face account and read-scoped access token
• At least 2.5TB of free local NVMe per node for FP8, or a shared volume all nodes can mount
• Docker with the NVIDIA container toolkit on each node, if running the containerized SGLang path

Check the Qwen Hugging Face organization page for the confirmed repository name before running any download command; it had not appeared as of this post's publication date. Once it does, the download pattern follows the same shape as every recent Qwen release:

python3 -m pip install -q huggingface_hub hf_transfer
export HF_TOKEN="hf_your_token_here"

nohup env HF_HUB_ENABLE_HF_TRANSFER=1 HF_TOKEN="$HF_TOKEN" \
  python3 -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/[REPO-NAME]', max_workers=16)" \
  > /tmp/dl.log 2>&1 &

HF_HUB_ENABLE_HF_TRANSFER=1 matters at this size: a 2.4TB download at unaccelerated throughput can take hours, while accelerated transfer typically finishes in under an hour on a multi-gigabit connection, the same speedup a 1.56TB Kimi K3 checkpoint sees under the identical flag.

Multi-Node Launch Commands (Tensor + Expert Parallelism)

For a 3-node, 24-GPU H200 FP8 deployment with vLLM, using tensor parallelism across the full cluster and expert parallelism for the MoE layers:

vllm serve /data/models/qwen38max \
    --served-model-name Qwen/Qwen3.8-Max \
    --tensor-parallel-size 24 \
    --enable-expert-parallel \
    --quantization fp8 \
    --gpu-memory-utilization 0.9 \
    --max-model-len 32768 \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code

For a single 8-GPU B300 node, no multi-node networking is required:

vllm serve /data/models/qwen38max \
    --served-model-name Qwen/Qwen3.8-Max \
    --data-parallel-size 8 \
    --enable-expert-parallel \
    --quantization fp8 \
    --gpu-memory-utilization 0.9 \
    --max-model-len 32768 \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code

The equivalent SGLang launch for the 3-node H200 configuration:

python -m sglang.launch_server \
    --model-path /data/models/qwen38max \
    --tp 24 \
    --ep-size 24 \
    --nnodes 3 \
    --node-rank $RANK \
    --dist-init-addr $HEAD_IP:20000 \
    --quantization fp8 \
    --host 0.0.0.0 \
    --port 30000 \
    --trust-remote-code

--trust-remote-code is worth keeping in every command above until vLLM and SGLang confirm native Qwen3.8-Max support in a stable release; it's a safe fallback that costs nothing once native support ships.

Validating the Deployment

Once the server is up, confirm it's serving correctly before routing production traffic:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen/Qwen3.8-Max",
        "messages": [{"role": "user", "content": "Explain sparse MoE routing in three sentences."}],
        "max_tokens": 512
    }'

Then check VRAM usage matches expectations with nvidia-smi across every GPU in the cluster, not just the head node. On a 24-GPU H200 deployment, each GPU should show roughly 100GB of resident weights plus KV cache overhead; materially less on any single GPU usually means a rank failed to load its shard. Watch throughput with nvidia-smi dmon -s pum -d 5 and check the framework's metrics endpoint (/metrics on vLLM) for TTFT, ITL, and KV cache saturation once real traffic starts flowing.

Qwen3.8 Max vs Kimi K3 vs DeepSeek V4: Size, Cost, Benchmarks

Putting size, hardware, and reported benchmarks side by side:

Kimi K3 remains the largest confirmed open-weight model by parameter count and the one with a real, measured checkpoint size. Qwen3.8-Max is the second model in that >2T tier and, notably, the first time Alibaba has open-sourced a Qwen-Max-class flagship at all, a real departure from its prior practice of keeping the Max tier API-only. DeepSeek V4 remains the accessible entry point for teams that don't need the extra scale: a single 4-GPU H200 node gets you a production endpoint, versus a multi-node cluster for either of the larger two.

If your workload doesn't strictly need a 2T+ parameter model, DeepSeek V4's smaller footprint is worth weighing against the operational complexity of standing up a multi-node MoE cluster. If it does, budget for the larger cluster from day one rather than discovering mid-provisioning that your node count is 144GB short of fitting the weights.

Spheron GPU Pricing for Qwen3.8-Max Workloads

Live pricing from the Spheron GPU offers API, read 08 Aug 2026. Multi-GPU cluster rates are calculated as the lowest available per-GPU rate times GPU count, an approximation since actual bundle offers vary by region and availability.

At current API pricing of $2/M input and $6/M output tokens, a 24x H200 on-demand cluster at ~$116/hr breaks even against the API somewhere in the range of tens of millions of tokens per month, depending on your input/output mix; run the exact math against your own traffic before committing to self-hosting at this scale. Self-hosting only pays off once volume is high enough to amortize a genuinely large fixed cost, and a cluster this size is not the place to guess.

Pricing fluctuates based on GPU availability. The prices above are based on 08 Aug 2026 and may have changed. Check current GPU pricing → for live rates.

Troubleshooting Multi-Node Qwen3.8-Max Deployments

OOM at roughly 90%+ of weight loading: The clearest sign your cluster is undersized relative to the real checkpoint, not just the naive estimate. This is exactly what happened when 8x B200 (1,536GB) tried to load Kimi K3's real 1,560.94GB checkpoint against a smaller naive estimate. If you're on a configuration close to the naive FP8 estimate (a single 8x B300 node, for instance), move to the next tier up rather than fighting --gpu-memory-utilization down further.

Process group hangs at multi-node startup: Verify every node has a distinct rank (0 through N-1, no duplicates) and that the full TCP port range NCCL and Gloo need is open between nodes. A partial firewall rule causes a hang at the barrier rather than a clear error.

NCCL timeout across nodes: Set NCCL_TIMEOUT=1800 and NCCL_DEBUG=INFO to diagnose. NVLink-connected SXM5/SXM6 nodes avoid most of this; PCIe-connected multi-node setups are slower to negotiate and more prone to timing out on the default window.

Model class not recognized: Add --trust-remote-code and confirm you're on the latest vLLM or SGLang release. A brand-new architecture on launch week routinely needs a framework version bumped within days of the weights landing, not months.

Tensor-parallel size doesn't evenly divide your GPU count: Use a GPU count that cleanly divides across your chosen TP/EP configuration. 24 GPUs divides evenly by 2, 3, 4, 6, 8, 12, and 24, which is part of why the 3-node H200 configuration is a practical target instead of an awkward one.

Suspiciously tight KV cache headroom after weights load: If you provisioned close to the naive FP8 estimate (single-node B300, for example) and see almost no room left for KV cache, that's the overhead margin problem, not a misconfiguration. Move to the next cluster tier rather than trying to reclaim memory with lower --max-model-len alone.

Qwen3.8-Max's real checkpoint size won't be confirmed until the open weights land the week of August 10, 2026, but the naive 2.4TB FP8 estimate already rules out anything smaller than a multi-node H200 or B300 cluster. Size early and leave headroom.

H200 SXM5 on Spheron → | Spheron B300 instances → | View all GPU pricing →