H200 vs MI325X Rental Price: Which Wins for Inference? (2026)

MI325X's spec sheet looks like the easy winner: 256GB of HBM3e against the H200's 141GB, and a higher rated memory bandwidth too. But real multi-model inference benchmarks tell a different story once you measure tokens per second per chip instead of reading a datasheet, and the H200 vs MI325X rental price gap only makes sense once you know which chip actually wins the workload you're paying for. This post works through the spec comparison, the decode-bound throughput numbers across four real models, and what each GPU costs to rent right now.

TL;DR: H200 vs MI325X Rental Price and Performance

H200 wins most decode-bound inference by 24-121% more tok/s/chip at lower cost per token, and Spheron rents it on-demand now; MI325X isn't in Spheron's catalog, so it only pays off on specific MoE models past a 100-second latency threshold.

• Memory: MI325X carries 256GB of HBM3e at 6TB/s against H200's 141GB at 4.8TB/s.
• Throughput: H200 delivers 103-121% more tok/s/chip than MI325X on Qwen 3.5 397B-A17B, and 24-67% more on MiniMax M2.7, at lower cost per million tokens.
• MI325X wins: Llama 3 405B FP8 with vLLM, and DeepSeek V3 670B MoE past 100 seconds latency, at roughly 20% better cost per dollar.
• Spheron: rents H200 SXM5 at $5.54/hr on-demand ($3.35/hr spot) as of 27 Oct 2026; MI325X isn't in the catalog. Check live pricing.

H200 vs MI325X: Memory Capacity and Bandwidth Compared

MI325X is the larger, higher-bandwidth part on paper. It's built on the same CDNA 3 compute die as its sibling MI300X, carrying 256GB of HBM3e running at 6TB/s, with TDP rising to 1,000W from MI300X's 750W to push that extra memory. H200 sits on the Hopper architecture with 141GB of HBM3e at 4.8TB/s, a 76% capacity and 43% bandwidth increase over its own predecessor, the H100, but still smaller and nominally slower than MI325X's package on both axes.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 27 Oct 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

If the comparison stopped at this table, MI325X would look like the stronger buy for any memory-bound job: more capacity, more rated bandwidth, same compute lineage as a chip AMD already ships at scale. For AMD's own MLPerf Inference v5.0 results, MI325X hit 1.5K TFLOPS prefill and 1.4K TFLOPS decode on Llama 2 70B on that same CDNA 3 silicon, numbers that look entirely competitive in isolation. The spec sheet, though, doesn't tell you what happens once a production serving stack is actually scheduling requests against that memory.

Decode-Bound Inference: Why Bandwidth Wins the Argument

Most production LLM serving spends the majority of its wall-clock time in the decode phase, generating one token at a time, reading the full weight set and KV cache from memory on every step. As SemiAnalysis puts it in its AMD vs NVIDIA inference benchmark analysis, "the decode phase of inference tends to be memory bandwidth bound." That's why this comparison is framed around bandwidth and not raw capacity: a GPU with more VRAM than your model needs doesn't make decode faster, but a GPU that moves weights and cache in and out of memory faster does.

That principle is also why MI325X's rated 6TB/s edge over H200's 4.8TB/s doesn't settle the argument on its own. Peak memory bandwidth on a spec sheet is the ceiling a workload could theoretically reach, not what a real serving stack actually extracts from it. How much of that ceiling gets used depends on kernel efficiency, scheduler overhead, KV cache paging, and how mature the attention and GEMM kernels are for a given architecture, which is exactly where the benchmarks below start to diverge from the spec table above. Capacity matters for a different reason entirely: it decides whether a model fits on one GPU at all, not how fast that GPU serves it once it does.

Real Benchmarks: Where MI325X Wins and Where H200 Wins

The clearest picture comes from InferenceX's multi-model head-to-head benchmarks, which hold sequence length and precision constant across both chips rather than quoting a single cherry-picked run.

On Qwen 3.5 397B-A17B (FP8, agentic-traces, 8K/1K context), H200 delivers 13,404 tok/s/chip against MI325X's 6,596 tok/s/chip at roughly 57 tok/s/user interactivity, with H200 costing about $0.03/M tokens against MI325X's $0.05/M tokens. Depending on the interactivity target measured, H200 delivers 121% more tok/s/chip than MI325X on this model, and runs about 40% cheaper per million tokens.

On MiniMax M2.7 (FP4, 8K/1K context), H200 delivers 3,353 tok/s/chip at $0.10/M tokens against MI325X's 2,699 tok/s/chip at $0.11/M tokens, measured at 34 tok/s/user interactivity.

MI325X isn't losing everywhere. On Llama 3 405B running FP8 with vLLM, MI325X consistently outperforms H200, as well as MI300X and H100, the one dense-model case in this set where AMD's larger memory pool and bandwidth translate directly into a throughput win. And on DeepSeek V3 670B, a mixture-of-experts model, H200 wins decisively below 100 seconds of latency, but MI325X achieves roughly 20% better performance per dollar above that threshold.

Read across all four models, the pattern isn't "AMD vs NVIDIA," it's dense versus MoE and tight versus loose latency targets. H200 wins the workloads most production APIs actually run: dense models at interactive latency. MI325X wins a specific MoE case and a specific dense exception, both of which come with a catch. DeepSeek V3's MI325X advantage only shows up once you accept triple-digit-second latency, which is well outside what a chat or agent API typically promises a user. And Llama 3 405B at 405 billion dense parameters is large enough that few teams are running it in a latency-sensitive production path to begin with.

Software Maturity: CUDA vs ROCm for Production Serving

Part of why H200 wins workloads where its raw spec sheet is behind is software. CUDA has 15-plus years of library depth behind it: TensorRT-LLM, cuDNN, and NCCL are tuned specifically for Hopper's Transformer Engine, which is what lets production serving stacks extract a larger share of H200's rated bandwidth than a less mature stack extracts from MI325X's larger, faster-on-paper memory. ROCm has closed real ground, with PyTorch, vLLM, and SGLang all carrying official support on AMD's CDNA hardware, but TensorRT-LLM has no ROCm equivalent, and teams moving to AMD should budget extra engineering time for kernel-level tuning rather than assuming a drop-in swap. Our ROCm vs CUDA GPU cloud guide walks through the full framework compatibility matrix and where ROCm still falls short on standard inference stacks.

This gap is also why MI325X is one of exactly two chips, alongside H200, named directly in a January 2026 US export control rule that moved China-bound exports to case-by-case review, capping shippable volume at roughly half of what an exporter ships to its US customers. Both chips sit inside the same regulatory bracket, which keeps neocloud supply for either one tighter than for hardware outside that rule, and is a real-world availability constraint independent of anything in the benchmarks above.

H200 vs MI325X Rental Price Right Now

Spheron's own pricing page lists NVIDIA H200 on-demand and spot, and AMD MI300X on-demand, but MI325X isn't in the catalog at all. H200 SXM5 runs $5.54/hr on-demand and $3.35/hr spot on Spheron as of 27 Oct 2026, billed per minute with no reserved commitment required.

MI325X's rental price has to be sourced from third-party neoclouds instead. Our MI325X pricing breakdown puts on-demand rates at roughly $2.25-$3.31/GPU-hr across providers, with TensorWave anchoring the low end near $2.25/hr. That median has risen about 44% since July 2025, from roughly $2.25/hr to roughly $3.24/hr, as the export-control supply squeeze above tightened reserved capacity, the opposite direction MI300X pricing moved over the same window.

Put the throughput numbers next to the rental rates and the decision gets simpler for most teams: H200 is both faster and cheaper per token on the dense and FP4 models most production APIs run, and it's actually rentable on Spheron today rather than something you have to shop across neoclouds to find. If your model is specifically MoE at a latency target north of 100 seconds, or dense at 405B parameters, MI325X's own numbers above are worth pricing out directly. And if 256GB on a single card is a hard requirement your model can't avoid, MI300X's 192GB on Spheron is the closer available AMD option, though it still falls short of MI325X's full memory tier.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 27 Oct 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

Most production LLM serving is decode-bound and dense, which is exactly where H200 wins on both tokens/sec/chip and cost per million tokens. Spheron rents H200 SXM5 on-demand and spot with per-minute billing, no reserved contract required.

H200 GPU pricing →