GPU as a Service (GPUaaS) means renting GPU compute from someone else's data center instead of buying the cards yourself. You pick a GPU, pay by the hour (or minute, or token), and hand it back when the job ends. Most AI teams run this way now, from solo developers fine-tuning on one RTX 4090 to labs training on thousands of H100s. This guide covers how GPUaaS works, the pricing models, the provider types, and when renting is the wrong call.
TL;DR: What Is GPU as a Service?
- Definition: GPU as a Service is renting GPU compute by the second, minute, hour or token from a provider that owns the hardware, power and networking.
- Price: an H100 on Spheron costs $2.75/hr on-demand as of 20 Oct 2026; AWS lists one H100 (p5.4xlarge) at $6.88/hr in us-east-1, per September 2026 rates.
- Six pricing models: on-demand, spot, reserved, bare metal, serverless per-second, and per-token inference APIs.
- Four provider types: hyperscalers, neoclouds, GPU marketplaces like Spheron, and serverless inference platforms.
- Rent vs buy: GPUaaS wins unless GPUs stay 70-90% utilized for years. Compare live GPU pricing before committing.
How GPU as a Service Works
The provider buys GPU servers, racks them in a data center with enough power and cooling (an 8x H100 server draws around 10kW), and connects them with high-bandwidth networking. A control plane on top lets you provision a GPU, attach storage, and get SSH or API access in minutes.
What you actually receive depends on the layer you rent:
| Layer | What you get | Who manages the OS and drivers | Typical use |
|---|---|---|---|
| Bare metal | A whole physical server, often 8 GPUs | You | Large training runs, steady inference fleets |
| GPU virtual machine | One or more GPUs passed through to a VM | You, on a provider image | Fine-tuning, dev boxes, single-node inference |
| Container or serverless | A container that scales to zero | Provider | Bursty inference, batch jobs |
| Inference API | An HTTP endpoint for a hosted model | Provider | Apps that just need tokens or images out |
The lower the layer, the cheaper the GPU-hour and the more work you do yourself. An inference API hides every GPU decision from you and charges for that convenience in its per-token rate. If you are weighing those two ends against each other, the GPU rental vs inference API cost breakdown runs the numbers.
GPU as a Service Pricing Models
GPUaaS pricing has more shapes than ordinary cloud compute, because GPU supply is scarce and providers price that scarcity in different ways.
| Pricing model | How you pay | Reliability | Best for |
|---|---|---|---|
| On-demand | Per hour or minute, no commitment | Guaranteed while running | Most work, anything you cannot restart cheaply |
| Spot | Per hour, discounted, can be reclaimed | Interruptible | Checkpointed training, batch inference, experiments |
| Reserved / committed | Fixed rate for 1-36 months | Guaranteed capacity | Steady production inference, long training programs |
| Bare metal | Per server-hour or monthly | Single tenant | Multi-node training, strict isolation needs |
| Serverless | Per second of active GPU time | Cold starts on scale-up | Spiky traffic, workloads idle most of the day |
| Per-token / per-image | Per million tokens or per output | Provider SLA | Apps calling a standard open model |
A few patterns hold across providers:
- On-demand and spot track different pools. On Spheron, H100 runs $2.75/hr on-demand and $2.21/hr spot as of 20 Oct 2026. Do not assume the order between them, because on a marketplace the two pools move separately. Spot capacity can be reclaimed without notice, so pair it with checkpointing. The spot instance arbitrage guide covers how teams run real work on it.
- Reserved pricing is a bet on your own utilization. A 12-month reservation only pays off if the GPUs stay busy. The serverless vs on-demand vs reserved comparison shows where each crosses over.
- Hyperscalers charge a premium per GPU. AWS sells a single H100 as the p5.4xlarge, which trackers list at $6.88/hr in us-east-1 as of September 2026, and the 8-GPU p5.48xlarge at $55.04/hr, the same $6.88 per GPU. Single-GPU sizes of the newest cards often come later than the 8-GPU bundles, if at all.
- Egress and storage add up. Some clouds charge per GB of data leaving the platform. At AWS's $0.09/GB, moving a 2TB checkpoint out costs about $180, more than a day of single-H100 time on most marketplaces.
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 20 Oct 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.
Types of GPU as a Service Providers
"GPUaaS provider" covers very different companies. Grouping them by how they get their GPUs explains most of the differences in price and availability.
Hyperscalers
AWS, Microsoft Azure, Google Cloud and Oracle Cloud rent GPUs next to everything else they sell. The draw is integration: IAM, VPCs, managed Kubernetes and your existing enterprise agreement. The cost is price and access. GPU-hours run well above marketplace rates, the newest GPUs often need a quota request or capacity reservation, and new generations usually arrive as 8-GPU instances first.
Neoclouds
CoreWeave, Lambda, Nebius, Crusoe and similar companies are clouds built around GPUs from day one. They buy large fleets, wire them with InfiniBand, and sell mostly on contracts, with some on-demand capacity. They are strong for multi-node training clusters. Our explainer on how neoclouds run GPU as a Service covers their business model from the supply side.
GPU marketplaces and aggregators
Marketplaces do not own a single fleet. They list capacity from many data centers and route you to whichever has the GPU you want. Spheron aggregates 5+ providers behind one console and one bill, so if one partner runs out of H100s, another usually has them. Vast.ai runs a similar model with a heavier mix of individual hosts. Prices are usually the lowest in GPUaaS because partners compete for your workload, and the trade-off is that rates move with supply.
Serverless and inference platforms
Modal, Baseten, Together AI, Fireworks and Replicate sit one layer up. You deploy a container or call a hosted model and pay per second or per token. They suit traffic that is idle most of the day. At sustained load, the per-second premium usually makes a dedicated GPU cheaper.
Regional and sector providers
Data residency rules created regional GPUaaS markets. India has providers like Yotta, E2E Networks and Neysa, covered in our GPU cloud India guide. Telcos are also turning edge sites into GPU capacity, which our GPU as a Service telecom guide breaks down.
For a ranked list of individual companies with prices, see the top cloud GPU providers comparison.
When GPU as a Service Makes Sense (and When It Doesn't)
Renting wins when your demand is uneven. Buying wins when GPUs would sit at high utilization for years: our analysis puts the point where owning starts to pay off at a fleet held at 70-90% utilization continuously, by around year three, once you count power, cooling, colocation, networking and the engineer who keeps the cluster healthy. The GPU rent vs buy TCO analysis works through that math for H100 and B200.
GPUaaS is the better fit when:
- Your workload is bursty. A two-week training run, then three weeks of evaluation on one GPU. Owning 64 GPUs for that is idle money.
- You need hardware you cannot get. B200 and B300 lead times for direct purchase run months. On a marketplace you can be on one today.
- You are not sure which GPU you need. Rent an RTX 5090, an H100 and an H200 for an afternoon each and measure tokens per dollar before you commit to anything.
- Generations move fast. A GPU bought today depreciates against next year's card. Renting moves that risk to the provider.
Buying (or a long reservation) is the better fit when:
- The fleet runs flat out for 3+ years. Steady production inference at high utilization is the classic case.
- Compliance demands physical control. Some regulated workloads need hardware in your own cage. Even then, single-tenant bare-metal GPUaaS in a named region often satisfies the same auditors. Check our list of SOC 2 compliant GPU cloud providers.
How to Choose a GPU as a Service Provider
Run every shortlist through the same six checks:
- Price per useful output, not per hour. A faster GPU at a higher rate often costs less per token or per training step. Benchmark your own workload.
- Availability of the exact SKU. "H100" can mean PCIe, SXM or NVL, with very different bandwidth. Confirm the variant and whether single-GPU instances exist.
- Billing granularity. Per-minute or per-second billing matters when jobs are short. Hourly rounding on a 10-minute job is a 6x markup.
- Interconnect for multi-GPU jobs. For training across GPUs, ask about NVLink within the node and InfiniBand or RoCE across nodes.
- Hidden costs. Egress fees, storage, idle charges on stopped instances, and minimum commitments.
- Exit cost. Proprietary SDKs make it expensive to leave. Plain Docker and SSH keep you portable.
Spheron is built around those checks: bare-metal and VM GPUs from 5+ providers, per-minute billing, no egress fees, and both on-demand and spot tiers on one console. You can launch Spheron H100 instances or anything from an RTX 4090 to a B300 without a sales call.
GPU as a Service works best when you can compare live rates across providers and switch without rewriting your stack.
Spheron H100 → | Check B200 availability → | Compare GPU pricing →
Frequently Asked Questions
GPU as a Service (GPUaaS) is renting GPU compute from a provider by the hour, minute, second or token instead of buying the hardware. The provider owns the servers, power, cooling and networking. You get a GPU instance, a bare-metal server, a container endpoint or an inference API, and you pay only for what you use.
It depends on the GPU and the pricing model. An H100 on Spheron runs $2.75/hr on-demand, as of 20 Oct 2026. AWS lists its single-H100 p5.4xlarge at $6.88/hr in us-east-1 (the 8x H100 p5.48xlarge is $55.04/hr), per published rates in September 2026. Consumer-class cards like the RTX 4090 rent for $0.48/hr. Spot and reserved pricing change those numbers again.
IaaS is the general category of renting virtual machines, storage and networks. GPUaaS is the GPU-specific slice of it, and it has its own quirks: GPU supply is tight, prices move with partner inventory, interconnect (NVLink, InfiniBand) matters for multi-GPU jobs, and many GPUaaS providers sell bare metal and per-token APIs that general IaaS clouds do not.
There are four groups. Hyperscalers (AWS, Microsoft Azure, Google Cloud, Oracle) sell GPUs next to the rest of their cloud. Neoclouds (CoreWeave, Lambda, Nebius, Crusoe) are GPU-first clouds. Marketplaces and aggregators (Spheron, Vast.ai) pool capacity from many data centers. Serverless and inference platforms (Modal, Baseten, Together AI, Fireworks) sell GPU time per second or per token.
It can be, but check three things before you sign: the provider's audit reports (SOC 2 Type II, ISO 27001), where the data physically sits (region and data residency), and whether you get single-tenant bare metal or a shared virtualized GPU. For regulated workloads, dedicated bare-metal GPUs in a named region with disk encryption you control are the safest GPUaaS setup.
