LLM Fine-Tuning Cost 2026: API vs Renting Your Own GPUs

OpenAI is shutting down self-serve fine-tuning. Not immediately, but on a fixed, published timeline that runs through January 2027. That single fact changes the entire LLM fine-tuning cost conversation, because "just use the API" is no longer a stable long-term option for a growing share of teams. Meanwhile, renting your own H100s for the same job keeps getting cheaper. This post walks through what hosted fine-tuning actually costs in 2026 across the providers still offering it, what renting GPUs costs for the same jobs, and where the real breakeven point sits, with worked numbers instead of hand-waving. For the mechanics of running the job yourself, our complete guide to fine-tuning LLMs in 2026 and the VRAM sizing tables for LoRA, QLoRA, and full fine-tuning cover the setup this post assumes.

The API Fine-Tuning Market Just Changed

OpenAI is winding down its fine-tuning platform on a three-stage schedule. Organizations that had never run a fine-tuning job lost the ability to start one on May 7, 2026. From July 2, 2026, any organization without fine-tuned-model inference activity in the prior 60 days can no longer start new jobs. Then, on January 6, 2027, every remaining customer, active or not, loses the ability to create new fine-tuning jobs at all. Inference on models you've already fine-tuned keeps running until the underlying base model is deprecated, so nothing you've shipped breaks overnight, but the door on starting anything new is closing in stages (source).

OpenAI's own reasoning, from the wind-down announcement, is that better prompting has closed the gap: "Newer base models like GPT-5.5 are much better at following instructions and formats than prior models. Prompt-based approaches are now cheaper and faster, as such, we're seeing fewer use cases that require fine-tuning" (source). Rinat Abdullin, founder of AI consultancy BitGN, put it more bluntly: fine-tuning and vector-based RAG "are dying...too expensive, risky and time consuming...if compared to other approaches: context engineering, proper tool-use and quality control." Laurie Voss, head of developer relations at Arize, called the shutdown "a strong signal that fine-tuning isn't what the average AI engineer wants to do" (both quotes via the same source).

That framing is one read of the data. The other read is narrower and more useful for cost planning: fine-tuning isn't dying, the managed API version of it is shrinking, right as the underlying GPU compute it runs on keeps getting cheaper to rent directly. Those are two different curves moving in opposite directions, and the rest of this post is about where they cross.

What Hosted API Fine-Tuning Actually Costs in 2026

With OpenAI's platform narrowing, Together AI and Fireworks AI are the two full-service options left for open-weight models, and both price per token rather than per GPU-hour.

Together AI Pricing by Model Size and Method

Together bills fine-tuning per 1M tokens processed (dataset tokens multiplied by epoch count), tiered by model size and method, with a $4.00 minimum charge per job (source):

Premium open-weight models sit outside this table entirely. DeepSeek, GLM, Llama 4, Qwen3, and Kimi variants are billed on separate, much higher per-token schedules, often $6-25/1M tokens for LoRA SFT depending on the model. If you're fine-tuning one of those, check Together's current rate card before you budget; the standard table above doesn't apply.

Fireworks AI Pricing by Model Size and Method

Fireworks uses a similar structure with its own tier boundaries, and it prices fine-tuned model inference identically to the base model with no separate hosting surcharge once training finishes (source):

Notice Fireworks charges double for LoRA DPO versus LoRA SFT at every tier, since preference optimization means processing chosen and rejected completions through the model on every step instead of one target output. Together's DPO premium is much smaller, only about 1.1x its matching SFT rate at every tier, so the bigger cost jump on Together's table is method, not objective: full fine-tuning runs roughly 2.5x the price of LoRA at the same tier, whether you're doing SFT or DPO.

What's Left of OpenAI's Fine-Tuning API, and Until When

As of this writing, o4-mini is the only model still listed on OpenAI's standard fine-tuning platform. Training runs $100.00 per hour, flat, with no per-token training rate published. Inference on the resulting model costs $4.00 per 1M input tokens and $16.00 per 1M output tokens on the standard tier, or $2.00/$8.00 per 1M on the batch tier (source). GPT-4.1 and GPT-4.1 mini, which OpenAI offered for fine-tuning earlier in the wind-down window, no longer appear on the live pricing page at all. If your organization is still inside the eligibility window described above, confirm current model availability and pricing directly with OpenAI before committing a training budget, since the roster is visibly shrinking month to month.

That $100/hr training rate is worth sitting with for a second. It's roughly 20x Spheron's H100 SXM5 on-demand rate of $5.07/hr, and roughly 50x its H100 PCIe rate of $2.01/hr (source, fetched 2026-07-07). Even accounting for whatever multi-GPU infrastructure OpenAI runs behind that number, it's a hosted-convenience premium, not a compute cost.

Where Anthropic Fits (It Doesn't, Outside Bedrock Haiku)

Anthropic doesn't expose fine-tuning through its own API at all. The only supported path in 2026 is supervised fine-tuning of Claude 3 Haiku through Amazon Bedrock, generally available in the US West (Oregon) region (source). It's a fully managed workflow inside the Bedrock console, but the fine-tuned model isn't served on standard pay-per-token pricing afterward. You have to purchase Bedrock Provisioned Throughput separately before you can run inference against it, a fixed hourly commitment that most fine-tuning cost comparisons never mention. If Claude's behavior is what you need, budget for that Provisioned Throughput line item on top of the training job itself.

What Renting Your Own H100s for Fine-Tuning Actually Costs

Renting GPUs flips the pricing model entirely: you pay for GPU-hours, not tokens, and the method you choose (LoRA, QLoRA, or full fine-tuning) determines which GPU tier you need before it determines what anything costs.

LoRA vs QLoRA vs Full Fine-Tuning: Which GPU Tier You Actually Need

QLoRA quantizes the frozen base model to 4-bit and trains small adapter layers on top, which is why a 70B model fits on a single H100 80GB with room to spare. LoRA keeps the base model at BF16 in memory, so a 70B LoRA run needs roughly 159 GB, more than one H100 holds, pushing you to 2x H100 SXM5. Full fine-tuning stores gradients and Adam optimizer states for every parameter, and at 70B that's about 860 GB, which requires 11x H100 SXM5 with FSDP or ZeRO-3 sharding across the cluster. We covered the full VRAM math for every tier in our GPU VRAM requirements guide; the short version is that the jump from QLoRA to full fine-tuning is not incremental, it's an order-of-magnitude jump in both GPU count and cost.

On-Demand vs Spot H100 Rates: Spheron vs AWS vs the Broader Market

Live Spheron pricing as of 2026-07-07 puts H100 PCIe on-demand at $2.01/hr per GPU, H100 SXM5 on-demand at $5.07/hr, and H100 SXM5 spot at $2.94/hr; an 8-GPU SXM5 node runs as low as $2.54/GPU/hr in bulk (source). For context, AWS P5 H100 on-demand runs about $6.88/hr per GPU, roughly 1.4x Spheron's H100 SXM5 on-demand rate for the same GPU tier, and 3.4x Spheron's cheaper H100 PCIe rate if PCIe is enough for your job; see our AWS H100 pricing breakdown for the full hidden-cost accounting on EBS, egress, and EFA networking that stacks on top of that headline number. Spot pricing on a marketplace like Spheron functions as a real, consistently available billing tier rather than the scarce, high-interruption AWS spot pool, which matters if you're planning to checkpoint through a multi-hour training run to save 40-60% on the same job.

Real Training-Run Costs for 7B, 70B QLoRA/LoRA, and 70B Full Fine-Tuning Jobs

Three worked examples, using current Spheron on-demand rates:

Framework choice affects the time column more than people expect. Unsloth and Axolotl both claim meaningful throughput gains over vanilla Hugging Face Trainer on the same hardware; if you're picking a training stack before you rent anything, our Axolotl vs Unsloth vs TorchTune comparison tested all three head to head on real jobs.

The Breakeven Table: Where API Fine-Tuning Wins and Where Renting Wins

Run the same job through both pricing models and the crossover point isn't really about model size. It's about how many times you're going to run the job and whether you already have someone who can operate a GPU training pipeline.

Take the 7B QLoRA example above: 50K examples at 512 tokens average across 2 epochs is 51.2M training tokens. On Together AI's sub-16B LoRA SFT tier ($0.48/1M), that job costs $24.58. On Fireworks ($0.50/1M), $25.60. On Spheron, the same job costs $2.76. Renting is roughly 9x cheaper in raw compute dollars, and that gap holds at 70B QLoRA too: the same token volume on Together's 70-100B tier ($2.90/1M) or Fireworks' 16.1-80B tier ($3.00/1M) runs $148-154, against $20.10 renting an H100 PCIe directly.

\* Both rows share the same real $736 rental total, that's the actual bill the case study reported for its batch. The Together and Fireworks columns bracket the low and high end of the 50K-500K example range its 10 customers spanned, since their per-token bill scales with dataset size in a way the rental total doesn't.

Small, One-Off, or Infrequent Fine-Tunes: API Wins on Total Cost, Not Raw Compute

Here's the honest version of this argument: on pure GPU-dollar terms, renting wins almost everywhere in the table above, even for a single small job. Together and Fireworks pull ahead once you price in the time to write a training script, configure Unsloth or Axolotl, provision and babysit a GPU instance, and stand up an inference server afterward for a model you'll fine-tune once and rarely touch again. A few hours of engineering time at any reasonable hourly rate erases the $20-150 difference in the table fast. If your team has never run a training job and doesn't plan to make a habit of it, the API's real value isn't the per-token price, it's that you skip all of that entirely: upload a JSONL file, get a fine-tuned endpoint back.

The one scenario where the API can also win on raw dollars is 70B LoRA (not QLoRA) run once: needing 2x H100 SXM5 for 8-14 hours puts the rental cost in the same range as, or slightly above, the equivalent Together or Fireworks job. That's the real crossover point, not a vague "small jobs favor API" rule of thumb.

High Token Volume, Repeated Runs, or Multi-Customer Fine-Tuning: Renting Wins

Every advantage widens once you're doing this more than once, though not because rental cost stays flat while dataset size grows. Our case study on running 10 concurrent fine-tuning jobs documents an AI API company fine-tuning separate Llama 3.3 70B variants for 10 enterprise customers on two bare-metal 8x H100 servers, hitting 94% GPU utilization and landing at $73.60 per fine-tuning job, $736 total, 36% cheaper than running the same jobs sequentially and 73-91% cheaper than managed fine-tuning platforms. That $736 isn't dataset-size invariant: it already reflects a batch where the slowest job, a 500K-example legal-domain fine-tune at LoRA rank 128, took 17.2 of the run's 18 total hours, while the fastest job, a 50K-example customer-support fine-tune, finished in 4.1 hours. Ten smaller jobs run together would likely finish faster and cost less than $736, since the whole batch is bounded by whichever single job takes longest, not by the sum of all ten. What stays genuinely fixed is the per-server hourly rate, not the total bill.

Compare that to Together or Fireworks, which bill per token with no such ceiling. Run those same 10 jobs at the small end of the case study's 50K-500K example range and the API bill lands around $1,485-1,536; run them at the large end and it climbs to $14,848-$15,360, a straight 10x jump because the bill scales linearly with tokens processed, job by job, regardless of how much infrastructure is shared underneath. The rental total moves too as job size grows, just nowhere near that steeply, because parallel execution amortizes it against the batch's slowest job instead of summing every job's cost independently.

If your fine-tuning workload looks like "one variant per customer, indefinitely," renting isn't just cheaper, it's the only model where cost doesn't grow linearly with your own growth. For serving those variants afterward without keeping a dedicated GPU per customer, LoRA multi-adapter serving covers hot-swapping adapters on one base model instance.

The Costs Neither Option Advertises

Both pricing models hide costs that show up after you've already committed.

Data prep is unpriced on both sides. Cleaning, deduplicating, and formatting 50K-500K instruction-response pairs into the right chat template is the same amount of work whether you're uploading to Together or launching your own training script. Neither vendor's pricing page accounts for it, and it routinely takes longer than the training run itself.

Engineering time is the API's hidden subsidy and renting's hidden tax. The API's flat per-token price already has Together's or Fireworks' training infrastructure amortized into it; you're not managing CUDA versions, NCCL configuration, or FSDP sharding. Renting hands all of that back to you. If nobody on your team has run a multi-GPU training job before, budget real hours for the learning curve on top of the GPU bill in the table above.

Idle GPU risk only exists on the renting side, and only if you provision ahead of need. If you reserve a dedicated GPU expecting a steady stream of fine-tuning jobs and the volume doesn't materialize, you're paying for idle compute the API model never exposes you to, since Together and Fireworks bill per job regardless of your utilization. Per-minute billing on a marketplace like Spheron limits this risk, but it doesn't eliminate it if you provision for a scale you haven't hit yet.

Inference hosting is a separate cost on both sides that's easy to leave out of a training-cost comparison. Fireworks and Together serve your fine-tuned model at base-model inference rates with no separate hosting fee, which is a real advantage baked into their pricing. Renting means you also stand up and pay for inference infrastructure, whether that's a dedicated GPU running vLLM or a serverless endpoint. Bedrock Haiku goes a step further and requires you to purchase Provisioned Throughput before you can serve anything at all.

Which Path Actually Makes Sense for Your Team

If you're fine-tuning once, on a small open model, and nobody on the team has touched a training script before, use Together or Fireworks. The per-token markup is real, but it buys you zero infrastructure and a working endpoint in hours.

If you're fine-tuning more than a handful of times a year, working with 70B+ models, or building any kind of multi-customer or multi-variant pipeline, rent the GPUs directly. The math in the breakeven table isn't close at that point, and it gets more lopsided every time you repeat the job.

If you're currently on OpenAI's fine-tuning platform for anything other than o4-mini, the wind-down timeline means this decision is being made for you regardless of preference. Start pricing the alternative now rather than after your organization loses eligibility to start new jobs.

Renting is where the real savings sit once you're past the first fine-tune. Spheron's H100 SXM5 on-demand pricing starts well under what Together, Fireworks, or OpenAI charge per equivalent training hour, with no per-token markup and no minimum commitment.

Check H100 availability →

Pricing fluctuates based on GPU availability. The prices above are based on 7 Jul 2026 and may have changed. Check current GPU pricing → for live rates.