Azure H200 Pricing 2026: ND H200 v5 Cost vs Spheron

Azure's ND96isr H200 v5 lists at $110.24/hr on-demand, or $13.78/hr per GPU, per Vantage's Azure instance pricing. In the cheapest regions, East US 2, West US 2, and West US 3, CloudPrice shows the same instance at $84.80/hr, or $10.60/hr per GPU. Spheron's H200 starts at $3.70/hr on-demand per GPU ($3.31/hr spot), with no quota process and no reserved commitment. We covered Azure's H100 pricing in a prior post but stopped there deliberately: this one picks up where that left off, on the H200-specific SKU and its own pricing quirks.

TL;DR: Azure ND H200 v5 vs Spheron H200 (July 2026)

Pricing fluctuates based on GPU availability. The prices above are based on 6 Jul 2026 and may have changed. Check current GPU pricing for live rates.

What Is the Azure ND H200 v5 (ND96isr H200 v5)?

The ND96isr H200 v5 is Azure's H200-generation successor to the ND H100 v5 series, aimed at the same distributed training and high-bandwidth inference workloads. It's a single VM size, Standard_ND96isr_H200_v5, built around 8 NVIDIA H200 Tensor Core GPUs, 96 vCPUs, and 1850 GiB of RAM.

Specs: 8x H200 141GB, InfiniBand, and What Changed From NDv5 H100

Per Microsoft's own ND H200 v5 series documentation, the instance has:

• 8x NVIDIA H200 GPUs with 141GB HBM3e each (1128GB total)
• 96 vCPUs (Intel Xeon Sapphire Rapids)
• 1850 GiB RAM
• 900 GB/s NVLink between GPUs
• A dedicated, topology-agnostic 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand connection per GPU (3.2 Tb/s aggregate per VM), with GPUDirect RDMA support across VMs in the same scale set
• 1x 1024 GiB local temp disk plus up to 8 NVMe disks at 28 TiB each
• Max network bandwidth of 80,000 Mbps across 8 NICs

The generational jump from NDv5 H100 to NDv5 H200 is entirely in the memory subsystem. Microsoft's own docs put it plainly: the H200 offers "a 76% increase in High Bandwidth Memory over the H100 GPUs" and pushes bandwidth from 3.35 TB/s to 4.8 TB/s. Compute throughput is unchanged, since both generations run the same Hopper GH100 die. We break down the full architectural delta, including MLPerf benchmarks, in H100 vs H200 and the complete H200 datasheet.

That memory upgrade matters more than the spec sheet suggests. MLPerf Inference v4.0 shows the H200 delivering roughly 42% higher throughput than the H100 SXM5 on Llama 2 70B in offline mode, driven almost entirely by the extra bandwidth reducing memory-bound stalls during decode. Azure's own vCPU, NVLink, and InfiniBand topology carries over unchanged from the H100 generation, so the only real trade to evaluate here is price against that throughput gain.

Azure ND H200 v5 Pricing: On-Demand and Regional Deltas

There's no Azure-published sticker price for this instance visible on the standard pricing pages; the numbers below come from third-party trackers that scrape Azure's retail rate card region by region. That in itself is a signal: the H100 generation has a longer public pricing history, and the H200 SKU is newer and thinner on published rate options.

Per-GPU Cost Breakdown

CloudPrice's regional breakdown for Standard_ND96isr_H200_v5 on-demand Linux pricing spans a wide range:

The spread between the cheapest and highest listed region is more than double, on the same hardware. Vantage's baseline figure of $110.24/hr, the number most third-party comparisons cite as "the" Azure H200 rate, sits toward the expensive end of that range rather than the middle. If you're pricing out a real deployment, check your target region specifically rather than anchoring on a single headline number.

Pricing fluctuates based on GPU availability. The prices above are based on 6 Jul 2026 and may have changed. Check current GPU pricing for live rates.

Reserved Pricing Reality (and Why Azure Doesn't Publish an NDv5 H200 Rate)

Here's the part that's easy to miss if you're used to Azure's H100 pricing structure: as of this writing, neither Vantage nor CloudPrice list a 1-year or 3-year reserved rate for the ND96isr H200 v5 SKU. Azure's own reserved-instance pricing offer page doesn't surface an NDv5 H200 line item either. Compare that to the ND H100 v5 series, which has a full published discount curve: roughly 35% off with a 1-year commitment and 55% off with a 3-year commitment against its own on-demand rate.

That gap doesn't necessarily mean Azure won't offer H200 reserved pricing eventually, newer SKUs often lag their reserved-tier rollout by a few quarters. It does mean that today, every hour you run on ND96isr H200 v5 is billed at the on-demand rate shown above, full stop. There's no discount lever to pull short of negotiating a custom Enterprise Agreement rate directly with Microsoft, which isn't publicly quoted anywhere.

The same gap shows up in spot pricing. Vantage's listing for this SKU shows no distinct Low Priority VM tier, the spot field matches the on-demand rate exactly, which reads as "not offered" rather than "no discount." The ND H100 v5 series, by contrast, has a working Low Priority tier in the $2.25-$3.69/hr per GPU range. If your workload is spot-tolerant, that's a real capability gap between the two generations right now, not just a pricing one.

The Hidden Costs on Top of the Hourly Rate

The instance rate is the number everyone quotes, and it's the smallest piece of a real Azure bill once you're running production workloads. The line items are the same ones we documented for the H100 generation, because they're general Azure platform charges, not GPU-specific ones: outbound data transfer at roughly $0.087/GB for the first 5 TB/month out of East US, Premium SSD managed disks at $0.17/GB/month for the P30 tier (NDv5's local NVMe doesn't persist across stops), Azure Files for datasets shared across multiple nodes, Azure Monitor and Log Analytics for anything beyond default metrics, and a support tier if you need an SLA on response time.

Here's what that looks like for a single ND96isr H200 v5 node run 40 hours a week (roughly 172 hours/month) at the baseline on-demand rate, with 10 TB of monthly egress and a 2 TB Premium SSD for checkpoints:

The extras add about 7.5% on top of pure compute, roughly $1,419/month. That ratio holds or worsens as you add nodes, since egress and storage scale with checkpoint volume while the underlying compute discount curve stays flat (there isn't one, per the previous section).

Pricing fluctuates based on GPU availability. The prices above are based on 6 Jul 2026 and may have changed. Check current GPU pricing for live rates.

Azure ND H200 v5 vs Spheron H200: Direct Comparison

Per-GPU Hourly Cost

Spheron's H200 on-demand rate is $3.70/hr per GPU, live from the platform's own pricing API. Even against Azure's cheapest published region, that's still a 2.9x gap for the same NVIDIA hardware, with no egress fees or quota process on top.

Cost-Per-Million Tokens

Applying the MLPerf 42% throughput gain to the 3,000 tokens/second baseline we used for H100 SXM5 batch-8 FP8 inference in the cost-per-token benchmarks post gives an estimated ~4,260 tokens/second for Llama 2 70B FP8 on H200. That's a directional estimate, not a measured benchmark on this exact configuration, but it's grounded in Microsoft's and MLPerf's own published multiplier.

Formula: ($/hr / 3,600 sec) x (1,000,000 / tokens_per_sec)

Real throughput will vary with batch size, quantization, and serving framework; the relative gap between providers holds regardless of the absolute number you measure on your own workload.

1-Year TCO: 8x H200, 40 Hours/Week Training

Even at Azure's cheapest published region, the 1-year TCO is over 3x Spheron's on-demand cost for the identical 8x H200 workload, with none of Azure's ancillary charges. For a broader read on how this pattern holds across hyperscalers and their custom silicon alternatives, AWS Trainium 3 vs H200 and B200 walks through the same math against AWS's own accelerator stack.

Pricing fluctuates based on GPU availability. The prices above are based on 6 Jul 2026 and may have changed. Check current GPU pricing for live rates.

Azure Quota and Capacity Reality Check for H200

The quota mechanics for ND H200 v5 follow the same pattern documented for the H100 generation: Azure subscriptions start with zero ND-series quota, and you request an increase through Azure Portal under Subscriptions > Usage + Quotas. For H100, that process typically takes 1-4 weeks depending on the vCPU count requested and region.

H200 is the newer, less broadly deployed SKU, and Azure hasn't published the same volume of reserved or spot options for it yet, which is itself a proxy for how thin regional capacity still is. If your quota request goes through, that's a permission to request instances, not a guarantee that Standard_ND96isr_H200_v5 capacity actually exists in your target region at the moment you try to provision. SkuNotAvailable and OverconstrainedAllocationRequest errors are common on newer high-demand SKUs even after quota approval, and the regions with the widest pricing spread above (the "highest" tier at $130-169/hr) are frequently the ones with the tightest capacity, since demand outstrips supply there.

For teams that can't absorb a multi-week quota-and-capacity cycle before a training run starts, that friction is often the deciding factor, independent of price. Spheron's overview docs cover how the platform aggregates capacity across 5+ providers instead of a single region's inventory, which is the structural reason it can provision H200 without a quota queue.

When to Stay on Azure vs When to Switch

Stay on Azure if your training pipeline is already deeply wired into Azure Active Directory, Azure ML, or AKS, if you're spending down Enterprise Agreement credits that don't transfer to other providers, or if FedRAMP High or HIPAA certification is a hard compliance requirement for your workload. None of that changes because the GPU generation moved from H100 to H200. The Azure H100 pricing post covers the same stay-vs-switch tradeoffs in more depth if compliance or ecosystem lock-in is the deciding factor for you.

Switch to Spheron H200 if the ~3-4x per-GPU price gap is a real budget constraint, if you need capacity now rather than after a quota approval cycle, or if your training and inference stack is Docker/SSH portable (most PyTorch and vLLM setups are, with no proprietary SDK to strip out). At $3.70/hr per GPU on-demand versus Azure's $10.60-$13.78/hr, the difference on a 6-month, 8-GPU training project is roughly $145,000 versus $30,000-$40,000 for identical H200 hardware.

If you're weighing the move, the step-by-step migration guide covers auditing your current Azure spend and standing up your first workload elsewhere without a rewrite.

The H200 generation is a real upgrade over H100 for anything memory-bound, longer context windows, larger batch sizes, bigger models that don't need tensor parallelism to fit. Azure prices that upgrade at a roughly 12% premium over its own H100 instance and, for now, offers no reserved or spot discount path to bring the on-demand rate down. Whether that's worth it depends entirely on how much your workload benefits from the extra 61GB of HBM3e, not on anything Azure-specific.

If the quota wait and the 3-4x per-GPU premium on Azure's ND H200 v5 don't fit your timeline or budget, the same NVIDIA hardware is available on Spheron without either constraint.

Rent H200 GPU →