Buying LLM inference usually gets framed as a two-way choice: call a per-token API, or rent a GPU and run the model yourself. That framing skips the option most teams actually end up evaluating once volume gets real, which is reserving inference capacity from someone else's infrastructure, either through a hyperscaler's provisioned throughput product or a prepaid plan sold directly by an inference provider. There are three structurally different ways to buy LLM inference in 2026, each billed differently, and the break-even between them depends on how your traffic is shaped, not just how much of it there is.
TL;DR: Serverless vs Dedicated Inference, Which Pricing Model Fits Your Traffic?
Serverless suits bursty traffic, provisioned throughput and dedicated endpoints suit sustained volume on one committed model, and prepaid reserved plans suit sustained volume with a fixed price and no multi-year contract.
- Dedicated endpoints: Together AI's dedicated H100 80GB endpoint bills $5.49/hr per replica, and a B200 180GB replica runs $8.99/hr, billed whether or not you send traffic.
- Provisioned throughput: Azure OpenAI PTU requires a 15-PTU minimum purchase for Global/Data Zone deployments.
- Break-even: one worked example puts PTU break-even near 585M tokens/month against pay-as-you-go, and only if the reservation stays busy all month.
- Reserved plans: Spheron's managed inference plans are prepaid and fixed-price for the term, with Growth allowing a model switch at renewal. See our GPU cloud pricing comparison for the self-hosted alternative's rates.
The Three Ways to Buy LLM Inference in 2026
Every inference pricing page on the market reduces to one of three mechanics. The difference isn't cosmetic: it decides whether your bill tracks token volume, wall-clock time, or a monthly allowance you paid for in advance.
| Pricing Model | How It's Billed | Minimum Commitment | Example Providers |
|---|---|---|---|
| Serverless per-token | Metered per input/output token, per request | None | Together AI serverless, Fireworks AI serverless, DeepInfra |
| Provisioned throughput / dedicated endpoint | Per hour, per reserved unit of capacity, whether or not you send traffic | Hyperscaler minimum purchase (Azure PTU has a 15-unit floor); Together and Fireworks dedicated endpoints bill hourly with no published minimum term | Azure OpenAI PTU, Amazon Bedrock Provisioned Throughput, Together AI Dedicated Endpoints, Fireworks AI dedicated deployments |
| Prepaid reserved plan | Monthly token allowance paid up front, price fixed for the term | Plan-dependent: Spheron's Starter tier is month-to-month, Growth requires a 3-month term | Spheron managed inference plans |
That third row is the gap in almost every serverless-vs-self-hosted comparison online. It's a different commercial product from both a per-token API and a hyperscaler capacity reservation: you're buying a token allowance, not raw GPU-hours, and the price is fixed before you sign rather than metered as you go.
Worth separating up front: this post is about buying finished inference from someone else, under one of these three pricing models. If you're instead weighing whether to rent the underlying GPU and run your own serving stack, that's a fourth path with its own billing model (serverless GPU, on-demand, and reserved GPU rental), covered in our serverless vs on-demand vs reserved GPU billing guide.
Serverless Per-Token APIs: How the Billing Actually Works
You send a request to a shared endpoint, the provider schedules it against every other tenant's traffic on the same cluster, and you pay for the tokens that request consumed. Nothing is reserved on your behalf. That's the whole mechanism, and it's why serverless pricing pages read like rate cards rather than capacity plans: a dollar figure per million input tokens, a different one for output, sometimes a service-tier multiplier on top.
DeepInfra's pricing is a clean example of how granular that metering gets. Its base rate scales per model, and a Flex/Standard/Priority tier system sits on top of it: Flex bills at 0.8x the base rate for requests that can tolerate waiting on spare capacity, Standard is the default 1x rate, and Priority is a 1.5x premium that queues your traffic ahead of everyone on Standard. You choose the tier per request, not per account, so the same API key can run cheap batch jobs on Flex and latency-sensitive production traffic on Priority simultaneously.
The tradeoff is the one every serverless API makes: no idle cost, but no capacity guarantee either. If the shared pool is busy, your request waits behind other tenants', and a provider can change a model's per-token rate or deprecate the model outright without any contract obligating them to give notice. That's fine for workloads with headroom to absorb a slow response or re-point to a different model. It's a liability for anything with a latency SLA or a model your product is tightly coupled to, which is exactly the gap provisioned throughput and reserved plans exist to close.
Provisioned Throughput and Dedicated Endpoints: Azure PTU, Bedrock Model Units, Together, Fireworks
Teams generally leave serverless for one of two reasons: the per-token bill at their actual volume has grown past what a fixed capacity price would cost, or they need a latency guarantee a shared multi-tenant pool structurally can't promise. Provisioned throughput and dedicated endpoints solve both by selling you a slice of hardware instead of a stream of tokens.
Azure OpenAI Provisioned Throughput Units (PTU) are Microsoft's version. Each PTU is a fixed slice of throughput, and Microsoft requires a minimum purchase of 15 PTUs for Global or Data Zone deployments (whichever is larger of the rounded estimate or the minimum), with regional minimums that vary by model: an earlier published table lists 50 PTUs for gpt-4o and 25 for gpt-4o-mini. Sizing is model-specific: PTUs required equals normalized tokens-per-minute divided by input TPM per PTU, and Microsoft's own docs recommend benchmarking against real traffic rather than trusting that estimate alone.
Amazon Bedrock Provisioned Throughput sells the same idea in Model Units, with three commitment tiers per Model Unit per hour: no commitment, 1-month, and 6-month, the longer term always cheaper. AWS prices Cohere Command at $49.50/hr with no commitment, $39.60/hr on a 1-month term, and $23.77/hr on a 6-month term, per Model Unit, as a worked example. What it doesn't publish is how many tokens per minute one Model Unit actually processes; Bedrock directs buyers to their account team or AWS sales for that figure, so you can't size a reservation against an Azure PTU count or a dedicated endpoint's throughput from the public pricing page alone.
Together AI's Dedicated Model Inference bills per minute per running replica rather than per Model Unit: an H100 80GB replica is $5.49/hr, and a B200 180GB replica is $8.99/hr, with H200, GB300, and B300 listed as contact-sales. Only ready replicas bill; provisioning and degraded time doesn't. Together's own docs work the economics for you: one H100 endpoint running continuously costs about $132/day, roughly $3,950 over 30 days, and that number doesn't move whether you send one request or a million. Our Together AI alternatives roundup lines up other providers against that same dedicated-vs-serverless tradeoff.
Pricing fluctuates based on GPU availability and vendor rate changes. The Azure, Bedrock, and Together figures above reflect their most recently published rates and may have changed; check each provider's own pricing page for what you'd actually pay today.
Fireworks AI runs a comparable split: serverless per-token pricing for bursty traffic, and dedicated deployments billed hourly for a model pinned to hardware you don't share. Dedicated only pays for itself once sustained throughput clears what the per-token rate would have cost at that volume, a function of tokens per second, not request count. Our Fireworks AI alternatives guide breaks down where that crossover tends to land by model size.
Every one of these products shares a structural weakness: you're renting capacity sized to one model. Switch models and you're resizing or renegotiating the reservation.
Prepaid Reserved Inference Plans: A Third Model Hyperscalers Don't Sell
Hyperscalers sell provisioned throughput in abstract capacity units billed by the hour. A different commercial shape exists alongside it: a prepaid monthly token allowance at a price fixed for the term, sold directly by the inference provider rather than metered against a raw GPU-hour or a Model Unit. Spheron's managed inference plans are built this way.
Token accounting also differs from a straight per-request meter: output weighs 4x, fresh input 1x, and cached input 0.25x (0.03x on DeepSeek V4.1 Flash), so a long agent context reused across turns stays cheap as the raw token count climbs. Usage past the monthly allowance keeps working at the plan's rate, unused tokens expire monthly rather than rolling over, and prepaying a full quarter takes 5% off the term price. Capacity runs on reserved GPUs, never spot, with zero data retention, and models are served at the precision their maker publishes.
That reserved-not-spot distinction isn't a line item. We've measured why it matters: every 8-GPU spot instance type we used for inference testing was reclaimed at least once on 14 Oct 2026, and on 16 Oct 2026 one B200 region stopped five of our spot VMs in a single day. Serving inference on spot without redundancy means absorbing that reclaim risk on every request in flight, the opposite of what a team buying reserved capacity for a latency SLA wants.
What a reserved, redundant setup buys back shows up in failure tests, not uptime percentages. Running live failure tests on our four-node 8xB200 inference fleet through the production gateway on 16 Oct 2026, we killed a spot node mid-stream and the in-flight streams finished on the other copy after about 7 seconds with zero failures; killing a node mid-tool-call still left all 16 of 16 in-flight requests ending with one complete, valid tool call; and a gateway hand-over during the same test window dropped 0 of 832 requests. That's the practical shape of the 99.5%/99.9% uptime and p99 TTFT commitments the tiers above are sold on: a request in flight when a node disappears still finishes.
The honest limits: Spheron doesn't publish a per-token or per-plan price the way Azure, Bedrock, Together, and Fireworks publish their rate cards, so you can't size the cost from the page alone and have to request a quote, with a capped trial key available before signing. The model menu is narrower too, GLM-5.3, GLM-5.3-Flash, and DeepSeek V4.1 Flash, against the wide catalog a serverless router gives you. And the Growth and Dedicated floors (a 3-month term, 50B+ tokens a month) mean genuinely low or unpredictable traffic is still better matched to serverless per-token billing than to any reserved tier, Spheron's included.
Break-Even by Utilization and Traffic Shape, Not Just Volume
Most comparisons size the serverless-vs-reserved decision purely on monthly token volume. That's the wrong variable on its own, because a reservation only pays for itself when it's busy, and "busy" is a function of how your traffic is shaped across the month, not just how much of it there is in total.
A worked example on Azure PTU pricing makes the shape problem concrete: at 360M tokens a month, a reserved PTU setup ran about $3,900 a month against $2,400 for Standard pay-as-you-go and $1,200 for Batch, with the article's own break-even estimate landing around 585M tokens a month, and only if traffic kept that reservation busy the entire time. A separate example in the same breakdown priced a 1-year PTU commitment at $14,365 against $16,900 month-to-month at 1M requests a month, putting the cost of keeping flexibility at roughly $2,535 a month, with Standard at $5,400 and Batch at $2,700 for that same volume. That second figure is a third-party snapshot dated 30 Oct 2026, not Microsoft's own published rate, worth treating as directional.
Both examples share the same trap: the break-even math assumes flat, predictable utilization, and real traffic rarely is. A reservation sized for your average load gets overrun during a spike, pushing overflow onto serverless anyway, and sits idle paying its fixed rate during every trough. Together's per-minute-per-replica billing makes the idle side explicit: a replica bills continuously once ready, whether you're sending it one request a second or none. The same logic applies to a PTU allocation or a Model Unit, you're paying for the ceiling you provisioned, not the floor you actually use.
This is also where self-hosting enters the picture honestly. If your traffic is high and steady enough to make any reservation worthwhile, renting the GPU and running your own serving stack is a fourth lever, distinct from all three models above and the one with the most operational work attached. Our Spheron vs Together AI breakdown runs that crossover math for a 70B model, down to the sustained tokens-per-second a self-hosted Spheron H100 instance needs to clear first.
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 10 Nov 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing for live rates.
Model Deprecation and Lock-In Risk on Long Reservations
Almost no comparison of these pricing models mentions the risk that shows up only after you've signed: a long reservation ties you to one model, and that model doesn't get to stay current for free.
Azure PTU and Bedrock Provisioned Throughput both reserve capacity against a specific model or Model Unit type, not a model-agnostic pool of compute. Bedrock's three commitment tiers, no-commitment, 1-month, and 6-month, get cheaper the longer you commit, so the biggest discount carries the longest exposure to whatever happens to that model over six months. If the provider deprecates it or a newer model pulls ahead on cost and quality, you're stuck running the reservation you paid for or eating the cost of resizing mid-term. Together and Fireworks' dedicated endpoints carry the same exposure at the replica level: a replica is provisioned to serve one model, and newer hardware tiers for some SKUs are contact-sales only, which slows any move to a successor model.
Serverless per-token APIs don't have this problem in the same way, because the model is a parameter on the request, not a hardware commitment. Swap models and you change a string in your API call. That flexibility is a real reason some teams stay on serverless even past the volume where a reservation would be cheaper on paper.
Prepaid reserved plans sit in between. Spheron's Growth tier explicitly allows a model switch at renewal, which is a direct answer to the lock-in risk a PTU or Model Unit reservation doesn't address: you're still committing to a term, but not to one model for the life of that term. Dedicated runs custom and fine-tuned models on single-tenant nodes instead, so its lock-in question is answered differently, through a named engineer and a contract negotiated around the specific model you bring. Either way, that's a narrower guarantee than full serverless flexibility, since you can't swap mid-term, but it's a meaningfully different risk profile from a hyperscaler reservation that assumes the model you sized against is the model you'll still be running when the contract ends.
Decision Framework: Match the Pricing Model to Your Traffic
None of these three models is categorically better. Each wins for a specific traffic shape:
- Bursty or unpredictable volume, any size: serverless per-token billing. You pay only for what you use, and you keep the freedom to swap models without renegotiating anything.
- Sustained, predictable volume on a hyperscaler you're already committed to: provisioned throughput (Azure PTU, Bedrock Provisioned Throughput) or a dedicated endpoint (Together, Fireworks), sized with real traffic data against the provider's own minimums, not the rough estimate on their pricing page.
- Sustained, predictable volume where you want a fixed price and a latency commitment without a multi-year hyperscaler contract: a prepaid reserved plan. Spheron's Starter tier covers this at month-to-month terms; Growth and Dedicated suit higher, steadier volume where the 3-month term and 50B+ token floor are easy to clear.
- Volume that spikes hard above an otherwise flat baseline: a hybrid, a reservation sized to the baseline with serverless absorbing the overflow, rather than a single reservation sized to the peak.
- High enough sustained throughput that the per-GPU math wins outright: renting the GPU and running your own serving stack, the option every reservation above is implicitly priced against.
Size the decision against your actual traffic shape, not last month's total token count, and revisit it every time volume or the model you're running changes materially.
Whichever model fits your traffic, the model-lock-in question is worth asking before you sign, not after. Request a managed inference plan quote →
Frequently Asked Questions
You send a request to a shared, multi-tenant endpoint and pay per input and output token, with no charge when you're not calling the API. The provider decides how your request is scheduled against everyone else's, which is why serverless APIs are cheap at low, unpredictable volume but give you no guarantee about latency or capacity during a traffic spike.
Mainly two reasons: cost and predictability. Once volume is high and steady enough, a fixed price per unit of reserved capacity (a PTU, a Model Unit, a dedicated replica, or a prepaid token plan) can beat the metered per-token rate, and reserved capacity comes with the latency guarantees a shared serverless pool doesn't offer.
Provisioned throughput (Azure PTU, Bedrock Provisioned Throughput) reserves hardware capacity measured in abstract units and is priced per hour, with a required minimum purchase and a term tied to one model. A prepaid reserved plan, like Spheron's managed inference tiers, sells a monthly token allowance at a fixed price for the term instead of raw capacity units, and the better tiers allow a model switch at renewal.
