Spheron vs Together AI: GPU Rental vs Inference API Cost

Together AI crossed roughly $1 billion in annualized revenue in February 2026, up from about $618 million at the end of 2025, and closed an $800 million Series C in July 2026 at an $8.3 billion valuation (Sacra; TechCrunch). It got there running four different pricing models at once: serverless per-token inference, single-GPU Dedicated Inference Endpoints, multi-GPU GPU Clusters, and managed fine-tune hosting. That's a lot of surface area for a buyer to price against, especially once you've outgrown the per-token tier and are deciding whether to keep paying for API calls or rent the hardware yourself.

This is a direct comparison, not another Together AI alternatives roundup. If you want the fuller field of ten competitors, we've already covered that in Together AI alternatives. Here, we're putting Spheron's bare-metal GPU rental against every one of Together's four pricing surfaces, with current rates and worked math on where the crossover actually sits.

Together AI's Four Pricing Models: Serverless, Dedicated, and GPU Clusters

Together AI describes its platform as an "AI Acceleration Cloud" spanning inference, fine-tuning, and infrastructure (PR Newswire). In practice that phrase covers four separate products with four separate bills.

Serverless Per-Token Pricing (Model-by-Model Range)

Together's serverless tier is the default: call the API, pay per token, no infrastructure to manage. Rates vary widely by model. As of this writing, Llama 3.3 70B runs $1.04 per 1M input tokens and $1.04 per 1M output tokens, gpt-oss-20B is the cheapest listed model at $0.05/$0.20 per 1M in/out, and DeepSeek V4 Pro sits at $1.74/$3.48 per 1M ($0.20 for cached input) (Together AI Pricing). The cheapest listed rate ($0.05/1M, gpt-oss-20B input) and the priciest ($3.48/1M, DeepSeek V4 Pro output) are almost 70x apart, so "Together AI pricing" isn't one number, it depends entirely on which model you're calling.

Dedicated Inference Endpoints (Single-GPU, Managed)

One step up from serverless, Dedicated Inference Endpoints give you a single GPU running one model with predictable latency and no shared-tenancy noise. Together prices an H100 endpoint at $5.49/hr on-demand and a B200 endpoint at $8.99/hr on-demand (Together AI Pricing). You still don't get root access to the node, Together manages the serving stack, but you're paying an hourly GPU rate instead of a per-token rate.

Instant GPU Clusters: On-Demand vs Reserved

Together's GPU Clusters product (marketed as Together Instant GPU Clusters) is where per-token billing disappears entirely. You rent multi-GPU nodes by the hour, on-demand or reserved for a fixed term. Current rates: H100 at $5.49/hr on-demand or from $3.99/hr reserved for 7-30 day terms, H200 at $6.79/hr on-demand or from $4.55/hr reserved, and B200 at $9.95/hr on-demand or from $9.09/hr reserved (Together AI GPU Clusters). Pika CEO Demi Guo credits the product with "amazing training performance, expert support, and the ability to scale to meet our rapid growth," and Krea co-founder Victor Perez points to it for "real-time, high-quality image and video generation at scale" (Together AI GPU Clusters). Worth noting: Together's single-GPU H100 Dedicated Endpoint is priced identically to its 8-GPU H100 Cluster on-demand rate, both $5.49/hr per GPU. On B200, the two products actually diverge, the Dedicated Endpoint at $8.99/hr undercuts the Cluster's on-demand rate of $9.95/hr by about 10%.

Fine-Tune Hosting (Brief Mention)

Together also runs a separate managed fine-tune product, billed per 1M training tokens by model size and method (LoRA SFT, full SFT, LoRA DPO, full DPO), with a $4.00 minimum charge per job (Together AI Pricing). We've already broken down the full pricing tables and compared them against renting an H100 directly in LLM Fine-Tuning Cost 2026: API vs Renting Your Own GPUs, so we won't re-derive that math here. Short version: for anything beyond a single small job, renting wins on raw compute dollars, and it isn't close.

How Spheron's Bare-Metal GPU Rental Works

Spheron runs one pricing model instead of four: rent a GPU, get root access, pay per minute. There's no separate "endpoint" product and no serverless tier, you provision an instance and run whatever inference stack you want on it, including vLLM, SGLang, or a custom serving setup.

Under the hood, Spheron aggregates capacity from data center partners across multiple regions and matches your request to available inventory in real time, rather than operating a single vertically integrated fleet the way Together AI does. That's a structural difference worth knowing about upfront: Together owns and operates its own GPU infrastructure end to end, while Spheron functions as a marketplace layer over multiple providers. The tradeoff shows up as lower headline rates and more availability options on Spheron, against Together's tighter first-party control over the hardware stack. For H100, that means bare-metal PCIe from $2.01/hr and SXM5 from $4.06/hr on a single GPU, or as low as $2.54/hr per GPU on an 8-GPU cluster (Spheron GPU pricing, fetched 2026-07-12). Check current H100 pricing on Spheron for live rates, since GPU availability shifts these numbers daily.

Once provisioned, you SSH in and run whatever serving stack you want. Spheron's own vLLM server deployment guide covers standing up an OpenAI-compatible endpoint on an H100 in a handful of commands, which is the same API surface your application already expects if it's currently pointed at Together's endpoint.

Spheron vs Together AI: Cost Per GPU-Hour

If you've already decided to rent a GPU rather than pay per token, this is the comparison that matters. Spheron's on-demand rates come from live marketplace data; Together's come from its public pricing pages, both fetched 2026-07-12.

H100: Cluster Rate Head-to-Head

Spheron's H100 SXM5 cluster rate is 54% below Together's on-demand Cluster rate, and 36% below Together's best reserved rate, without asking for a commitment. The H100 PCIe rate goes further: at $2.01/hr, it's half of Together's cheapest H100 rate at any commitment length.

H200 and B200: Cluster Rate Head-to-Head

H200 is the cleanest win: Spheron's on-demand rate of $4.54/hr essentially matches Together's reserved floor, with no term lock-in. B200 tells a different story, and it's worth being straight about it. Together's cheapest B200 option, its $8.99/hr Dedicated Endpoint, is a few percent cheaper than Spheron's single-GPU B200 dedicated rate of $9.36/hr. Blackwell inventory is tighter across the market generally, and that shows up here. Where Spheron pulls ahead on B200 is spot: $5.34/hr per GPU on an 8-GPU cluster, 41% below Together's reserved floor, if your workload can tolerate a spot instance being reclaimed. For a broader market view beyond these two providers, see our 2026 GPU cloud pricing comparison covering 15+ providers.

Pricing fluctuates based on GPU availability. The prices above are based on 12 Jul 2026 and may have changed. Check current GPU pricing → for live rates.

Spheron vs Together AI: Cost Per Token

Cluster rates only matter if you've already committed to renting a GPU. The harder question for a lot of teams is whether to make that jump at all, versus staying on Together's serverless per-token tier. That's a throughput question, not a hardware question.

Worked Example: Llama 3.3 70B Serverless vs a Rented H100

A single H100 running vLLM with continuous batching handles roughly 400 tokens/sec at moderate concurrency on a 70B model, which is 34.56M tokens/day (throughput methodology). At Spheron's H100 PCIe on-demand rate of $2.01/hr, that instance costs $48.24/day flat, regardless of how many tokens you actually push through it.

Together AI prices Llama 3.3 70B at $1.04 per 1M tokens for both input and output. Because the input and output rates are identical right now, the prompt-to-completion ratio doesn't change the blended price the way it would if the two rates diverged, every token, prompt or completion, costs the same $1.04/1M on Together's side. At 34.56M tokens/day of pure output, ignoring prompt tokens entirely, Together's bill comes to $35.94/day, still cheaper than the $48.24/day GPU. Add prompt tokens at a typical 3:1 prompt-to-completion ratio and total token volume roughly quadruples to 138.24M tokens/day, pushing Together's bill to $143.77/day, about three times the flat GPU cost at that volume.

Where the Crossover Point Sits

Solving for the break-even point: $48.24 divided by $1.04/1M tokens is about 46.4M total tokens per day where the two costs meet. At a 3:1 prompt-to-completion ratio, completion tokens are a quarter of that total, so the crossover lands around 11.6M completion tokens per day, roughly 134 tokens/sec of sustained generation. Below that volume, Together's serverless tier is cheaper because you're not paying for GPU idle time between requests. Above it, the dedicated H100 wins, and the gap compounds daily since the GPU's cost is fixed while the API bill keeps scaling linearly with tokens processed. Our Together AI alternatives comparison walks through a similar crossover calculation using April 2026 pricing and lands in a comparable range; the exact number moves with whatever Together charges that month, but the shape of the curve doesn't change. For the general framework behind sizing this decision, see our inference cost economics playbook.

If your volume is well below that line and staying serverless is the right call but Together's rates feel high for your model, Fireworks AI is the closest serverless competitor worth a rate check before you commit to either.

When You Need Your Own GPU Cluster Instead of an Inference API

Signals You've Outgrown Serverless

• Sustained throughput above the crossover. If you're consistently running more than roughly 130-150 completion tokens/sec on a single model, per our worked example above, the dedicated GPU is already cheaper and the gap only grows.
• Fine-tuning on data that can't leave your environment. Together's managed fine-tune hosting runs on their infrastructure. If your training data has residency or compliance constraints, you need a GPU you control. See how to fine-tune LLMs in 2026 for the setup.
• Latency SLAs that cold starts can't meet. Serverless endpoints, including Together's, carry cold-start and queueing variance under shared load. Dedicated instances have none of that.
• Running more than one model on the same hardware. A rented cluster lets you co-host two or three smaller models on one node; a per-token API bills each model separately with no shared-infrastructure discount.

Signals a Managed API Still Makes Sense

• Low or bursty daily volume. If you're well under the crossover point, paying per token beats paying for idle GPU-hours.
• No one on the team has run GPU infrastructure before. The per-token markup buys you zero ops burden. Weigh that against the serverless vs on-demand vs reserved framework before deciding either way.
• You need access to many models without hosting each one. Together's catalog spans 200+ open models; standing up that many endpoints yourself isn't a realistic alternative for most teams.
• You're still prototyping. Committing to a GPU rental before you know your traffic pattern is how idle-compute cost creeps in.

Full Side-by-Side Comparison Table

Rates fetched 12 Jul 2026 from Together AI's pricing page, GPU Clusters page, and Spheron's live GPU offers API. Check current pricing → since both platforms update rates with availability.

Which One Should You Use

If your daily volume sits below roughly 130-150 tokens/sec sustained on one model, stay on Together's serverless tier. Together AI's own numbers back that logic up broadly: CEO Vipul Ved Prakash says customers moving workloads to open models on the platform routinely see 6-20x cost reductions versus closed-model APIs, and named customer Decagon reported cutting inference costs sixfold after switching (Tech Times). That's a real advantage over paying for a closed-model API, and it's independent of whether you eventually move off Together entirely.

Once you cross that threshold, or you're fine-tuning on data that can't leave your infrastructure, or you need predictable latency without cold starts, the GPU-hour math takes over. And if you've already decided to rent a GPU rather than pay per token, Together's own GPU products aren't the cheapest way to do it: Spheron's H100 and H200 rates undercut Together's Dedicated Endpoints and GPU Clusters at every commitment length, with per-minute billing and no term lock-in. B200 is the one tier where Together's Dedicated Endpoint holds a slight edge on-demand, so check both before committing if Blackwell is what you need.

If you're past the point where per-token billing makes sense, run the same worked math against your own token volume before committing to either platform.

Spheron H100 → | H200 on Spheron → | B200 on Spheron →

Start building on Spheron →