H100 vs MI300X Training Cost: Fine-Tuning Benchmark (2026)

Every public H100 vs MI300X comparison out there is either a spec sheet or an inference-serving benchmark, and nobody has published a timed, same-config fine-tuning run that answers what H100 vs MI300X training cost actually looks like in wall-clock minutes and real dollars, not marketed TFLOPS. So we ran one: the same 8B model, the same LoRA recipe, the same dataset, and the same epoch count, once on an H100 and once on an MI300X. The number a spec sheet can't give you showed up immediately: the MI300X job took longer and cost more, even though its hourly rate looked cheaper going in.

TL;DR: H100 vs MI300X Training Cost for the Same Fine-Tune

• Verdict: H100 won Spheron's 13 Sep 2026 Llama 3.1 8B LoRA benchmark over a Runpod MI300X on speed and spend, at near-equal final loss (0.91 vs 0.93). More in LLM fine-tuning cost: API vs renting GPUs.
• Wall clock (setup included): about 18 minutes on Spheron's H100 and 34 minutes on Runpod's MI300X for the same 375-step LoRA job on 13 Sep 2026.
• Cost: at 13 Sep 2026 rates, the 8B LoRA job cost about $0.89 on Spheron's H100 and $1.35 on Runpod's $2.39/hr MI300X (51% more), which Runpod listed Unavailable on 17 Sep 2026.
• Now on Spheron: MI300X is on-demand at $3.59/hr per GPU as of 17 Sep 2026, alongside H100 on the same account.

Pricing fluctuates based on GPU availability. Spheron's MI300X rate above is live as of 17 Sep 2026; the run costs use 13 Sep 2026 rates, and the Runpod MI300X rate reflects Thunder Compute's most recent published tracker and may have changed. Check current GPU pricing → for live rates.

The Test: Same 8B Model, Same Dataset, Same Epochs, Two GPUs

The design goal was to remove every excuse for an unfair comparison. Same model, same data, same LoRA settings, same effective batch size, same framework, same step count. The only things allowed to differ were the GPU underneath it, the software stack that talks to it, and one out-of-memory workaround on the MI300X leg, which split the same effective batch of 16 into smaller micro-batches (covered below).

Hardware and Where Each GPU Actually Came From

The H100 side is straightforward: a single H100 SXM5 80GB instance rented on-demand through Spheron, deployed from its live marketplace in the usual couple of minutes.

The MI300X side needed more legwork. We rented it on Runpod because Spheron's public pricing page, which is what we checked when setting up this run, listed only NVIDIA GPUs (H100, H200, B200, B300, A100 and the rest of that lineup), with no MI300X, no MI355X, nothing AMD. That's the reason this benchmark was assembled across two separate platforms instead of run side by side on one. Update, 17 Sep 2026: Spheron added the MI300X on 10 Sep 2026 ($3.59/hr per GPU on-demand), and Runpod's console showed its MI300X as Unavailable on 17 Sep 2026. That 10 Sep launch was on Spheron's marketplace, and the public pricing page we checked had not listed it yet. Spheron's MI300X cloud pricing covers the 1x and 2x instances it rents, so both halves of this test can now run on one account. Every MI300X result below still comes from the Runpod run.

As of 11 Sep 2026, Thunder Compute's MI300X tracker listed on-demand MI300X rates from $1.71/hr (TensorWave, quote-only) to $6.00/hr (Azure/Oracle), with Runpod at $2.39/hr as the lowest self-serve on-demand rate on that tracker (Runpod's console showed that MI300X as Unavailable on 17 Sep 2026, so it could not be deployed on that date). We rented from Runpod for exactly that reason: it was the option that let you spin up a single MI300X on-demand, by the hour, without a quota queue or a sales call, which is what a single-GPU LoRA job actually needs. Azure's Standard_ND96isr_MI300X_v5 is the instance most MI300X coverage benchmarks against, and it's worth knowing about even though we didn't rent it directly: Azure's Standard_ND96isr_MI300X_v5 (8x MI300X 192GB) reached general availability on May 21, 2024, with Hugging Face as launch customer, and it's sold as a full 8-GPU VM rather than a single-card rental, which makes it the wrong shape for a one-GPU fine-tuning job even before you get to price.

Pricing fluctuates based on GPU availability. Spheron's H100 and MI300X rates above are live as of 17 Sep 2026; the Runpod MI300X rate reflects Thunder Compute's most recent published tracker and may have changed, and Runpod's console showed its MI300X as Unavailable on 17 Sep 2026, so that rate could not be deployed that day. Check current GPU pricing for live rates.

MI300X's 192GB of HBM3 is close to 2.4x H100's 80GB, a gap the AMD MI300X vs NVIDIA H100 memory comparison covers in more depth for inference workloads. For an 8B model with LoRA adapters, that extra headroom doesn't do much: the whole point of LoRA is that the base weights stay frozen and only a small set of adapter weights and their optimizer states need memory, so an 8B run barely touches either GPU's ceiling. Memory capacity becomes the deciding factor at larger model sizes, which is a separate question from the one this test answers.

Training Config: LoRA Rank, Batch Size, Dataset, Epochs

We used torchtune's lora_finetune_single_device recipe against a Llama 3.1 8B base checkpoint, the same model AMD's own tutorial trains. Configuration:

• Base model: Llama 3.1 8B
• Method: LoRA, rank 16, alpha 32, dropout 0.05, applied to the attention and output projections
• Dataset: a 2,000-example subset of the Alpaca instruction dataset, held fixed across both runs
• Batch size: 4 per device with gradient accumulation 4 on H100; 2 per device with gradient accumulation 8 on MI300X after an out-of-memory error at batch size 4 (effective batch 16 on both)
• Sequence length: 2,048 tokens
• Epochs: 3, giving 375 total training steps at this effective batch size
• Precision: bf16

We picked LoRA over full fine-tuning specifically because it's the most common way teams actually adapt an 8B model on rented hardware, and because it keeps the job short enough to run twice, on two different clouds, in an afternoon. If you're choosing between LoRA and newer PEFT variants for a larger model, the DoRA, GaLore, PiSSA, and VeRA guide covers where those trade off against plain LoRA.

Software Stack on Each Side (CUDA vs ROCm)

The H100 box ran a standard CUDA 12.x PyTorch environment with torchtune installed via pip, the same setup most teams already have.

The MI300X box ran AMD's own recommended path. AMD's official ROCm AI Developer Hub ships a working Llama-3.1 8B LoRA fine-tuning tutorial using torchtune, ROCm 6.3, the rocm/pytorch-training:latest Docker image, config llama3_1/8B_lora_single_device, and the Alpaca dataset by default. We followed that recipe as closely as possible rather than inventing our own ROCm setup, since it's the path AMD itself points developers toward.

Wall-Clock Time and Training Cost: What We Actually Paid

The short answer: the H100 job finished in about 18 minutes and cost about $0.89; the MI300X job took about 34 minutes and cost about $1.35, roughly 51% more, even though Runpod's MI300X hourly rate looked 20% cheaper on paper that day. Job duration decided the winner, not the sticker price.

Time to First Step and Time to Completion

Time to first step is the part spec sheets never mention, and it's where the two platforms diverged the most before a single training step even ran.

On Spheron, the H100 instance came up with CUDA and the Python environment already usable, so getting from a fresh box to the first logged training step took about 3 minutes: pip-installing torchtune, downloading the base checkpoint, and starting the recipe.

On Runpod's MI300X, the ROCm 6.3 environment came from pulling AMD's rocm/pytorch-training:latest Docker image, a large image that isn't cached on a fresh instance. That pull, plus the same checkpoint download and recipe startup, pushed time to first step to roughly 14 minutes.

Total Spend and Cost per Epoch

Freeze the math to what both platforms billed on the day we ran this: Spheron's H100 on-demand rate was $2.97/hr on 13 Sep 2026, and Runpod's MI300X on-demand rate read $2.39/hr, per Thunder Compute's MI300X pricing tracker, though Runpod's console showed that MI300X as Unavailable on 17 Sep 2026. Those two figures are held fixed here so the totals below stay internally consistent; they aren't current prices, for those see the live H100 rate earlier in this post or the pricing page. At those frozen rates, the 18-minute H100 run cost about $0.89 total, or roughly $0.30 per epoch. The 34-minute MI300X run cost about $1.35 total, or roughly $0.45 per epoch, about 51% more per epoch than H100 despite MI300X's lower per-hour price.

That inversion is the entire point of running this as a timed job instead of a rate-card comparison. A pricing page tells you MI300X was 20% cheaper per hour that day. It can't tell you that the job took 89% longer, and that the longer job erased the cheaper rate and then some. Per-minute billing, which Spheron uses for its H100 instances, makes this kind of short job cheap to actually test; an hourly-minimum billing model would have rounded both runs up to a full hour and hidden the gap entirely.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 17 Sep 2026; the Runpod rate reflects Thunder Compute's most recently published tracker and may have changed. Check current GPU pricing directly with each provider for live rates.

Final Loss: Did Both Runs Converge the Same Way

Speed differences only matter if both runs actually produced a usable model. They did. H100 finished at a training loss of about 0.91; MI300X finished at about 0.93, close enough that neither run has a real quality edge over the other. The MI300X loss curve was noisier for its first roughly 20 steps, a direct consequence of the batch-size change described below, and it converged onto the same trajectory as H100 for the remainder of training. The cost and time gap documented above is a throughput and setup-friction story, not a model-quality one.

What Broke on ROCm (and What Didn't)

Environment Setup Friction

ROCm's documented gotchas for this exact tutorial are OOM errors requiring batch size reduction or activation checkpointing, and a Hugging Face token that must have Meta Llama checkpoint access approved before the job can start. We hit both. The Hugging Face gate needed clearing before either run could download Llama 3.1 weights at all, so it cost us setup time once, not twice, but it's a step a pure spec comparison would never surface. The OOM was more specific to the MI300X leg: our first attempt at batch size 4 failed with an out-of-memory error despite MI300X's 192GB of HBM3 dwarfing H100's 80GB, because the failure was in ROCm's activation memory handling during the backward pass, not in raw capacity. Dropping to batch size 2 with gradient accumulation doubled to 8 (keeping the effective batch size at 16) fixed it, which is the exact workaround AMD's own tutorial calls out.

Where Throughput Fell Short of the Spec Sheet

The per-step timing gap came in wider than the pure compute numbers suggest. In SemiAnalysis's independently run GEMM benchmarks, H100/H200 realized ~720 TFLOP/s BF16 against MI300X's ~620 TFLOP/s, a 14% gap despite MI300X's higher marketed spec (1,307 vs 989.5 TFLOP/s), and on FP8 GEMM the same benchmarks put H100/H200 at ~1,280 TFLOP/s against MI300X's ~990 TFLOP/s, a 22% gap. Our own end-to-end per-step gap ran closer to 33%, more than double the smaller of those two isolated GEMM numbers. That's consistent with what SemiAnalysis found once a full training loop is involved rather than a single matrix multiply in isolation: SemiAnalysis's Llama3 8B single-node training tests found H100 and H200 beating MI300X "by a big margin" on public stable AMD software releases. Attention kernels, memory-bound operations, and general kernel maturity all stack on top of the raw GEMM gap once you're running a real model instead of a synthetic benchmark.

Our own run was small enough, and short enough, that it didn't hit anything as severe as a broken PyTorch release. But the OOM workaround above is the same category of problem at a smaller scale: capability that exists on paper needing a manual fix to actually use.

What Ran Without Any Changes

The genuinely good news: the training recipe itself needed zero code changes between platforms. The same torchtune recipe, the same Alpaca dataset loader, the same tokenizer, and the same LoRA adapter definitions ran on both GPUs untouched. The YAML config changed in one place: the MI300X leg's batch-size workaround (batch size 2 with gradient accumulation 8, the same effective batch of 16). The container underneath also differed, a standard CUDA/PyTorch image on H100 versus AMD's rocm/pytorch-training:latest on MI300X. Torchtune abstracts the backend cleanly enough that porting a LoRA job between an NVIDIA and an AMD GPU is a Docker image swap and, in our case, one batch-size setting, not a rewrite. That's a real point in ROCm's favor even though the run itself came in slower: the friction was in setup and throughput, not in framework compatibility.

H100 vs MI300X Training Cost: Which One to Rent for Your Next Fine-Tune

For a LoRA-scale job on an 8B model, this run says H100 wins on both dimensions that matter: it finished faster and it cost less, despite the Runpod MI300X's lower advertised hourly rate that day. That tracks with the broader pattern SemiAnalysis found across its own training benchmarks: training performance per TCO is worse on the MI300X on public stable releases of AMD software, even where the raw rental price favors AMD.

That doesn't make MI300X pointless. Its 192GB per GPU is a real advantage the moment a model's weights and optimizer states stop fitting comfortably on one H100, at which point the alternative isn't "MI300X vs H100" but "one MI300X vs two H100s sharded with tensor parallelism," a different comparison with different economics. If you're training or serving something in that size class, the Best GPU for LLM Training guide and the GPU VRAM requirements guide walk through where that tradeoff actually flips.

For everything at or below the size we tested here, where the job fits comfortably on a single H100's 80GB, there's no memory advantage to trade against MI300X's slower wall clock and higher realized cost. If what you're after instead is a cheaper single-GPU alternative to H100 rather than a bigger-memory one, our RTX PRO 6000 96GB training benchmark against H100 reuses this exact H100 baseline and prices it against RTX PRO 6000 on the same LoRA recipe. AMD itself is clear that MI300X is a serious production accelerator; Victor Peng, AMD's president, has said "the AMD Instinct MI300X and ROCm software stack is powering the Azure OpenAI Chat GPT 3.5 and 4 services, which are some of the world's most demanding AI workloads". That's a real deployment at real scale, running on infrastructure and tuning most single teams won't replicate. On the stock tutorial path most teams will actually use, though, the numbers in this run favor renting the NVIDIA card.

On-demand H100 capacity through Spheron is available with per-minute billing and no long-term commitment, which is exactly the billing shape a short LoRA job like this one benefits from. If your next fine-tune needs AMD's memory headroom specifically, MI300X capacity is on Spheron now, as 1x and 2x on-demand instances with a 20-minute minimum runtime on 1x and 60 minutes on 2x, and Spheron's docs cover the deployment flow. Azure's ND MI300X v5 is the self-serve 8-GPU option; on Spheron, 8-GPU MI300X nodes are reserved capacity you request rather than launch.

If your next fine-tune fits on a single H100's 80GB, that's the config this benchmark says to rent; if it needs 192GB on one GPU, Spheron rents the MI300X on-demand from the same account.

On-demand H100 → | On-demand MI300X →