A rated boost clock is a number on a spec sheet, measured under conditions nobody runs in production. It is not the number you get at hour four of a continuous inference job. Here's the check that actually answers the question: rent a bare-metal GPU, run a continuous inference workload against it for four-plus hours, and log clock speed, power draw, and temperature with nvidia-smi the entire time. That gap between rated and sustained clock speed is exactly what a gpu performance consistency cloud check is built to catch, and it's invisible on a spec sheet. Below is the exact test setup to run against a rented GPU, a Spheron bare-metal H100 in the walkthrough here, laid against the sustained-load results other teams have already logged with the same method, so you can see what a healthy run looks like, what a bad one looks like, and what the gap between the two costs in tokens per second and dollars.
TL;DR: Does GPU Performance Consistency Hold Under Sustained Cloud Inference Load?
- The decay is fast. One cloud GPU case study saw an H100's clock fall from 1,755 MHz to between 450 MHz and 550 MHz within about a minute of sustained load.
- Heat triggers it, not tenancy. NVIDIA's TensorRT docs put the thermal throttle point near 85°C, on single-tenant bare metal too.
- Decay varies by card. Chips and Cheese saw an H100 dip to 1,395 MHz (80% of boost); an A100 SXM4 held 1,410 MHz.
- Power caps cost less than expected. Puget Systems cut four RTX 3090s' power limit 10-20% and kept over 95% of training throughput.
- Spheron: an H100 SXM5 bare-metal rental runs $2.98/hr as of 15 Sep 2026; bare metal makes clock and power logging possible. Check H100 pricing.
The Hypothesis: Does a Rated Boost Clock Hold for 4+ Hours of Continuous Inference?
Every GPU spec sheet lists a boost clock, and every cloud listing built on that GPU implies you get it. NVIDIA's own TensorRT performance documentation is blunt about why that's the wrong way to read the number: GPU clock frequency floats by default, sitting idle with no workload and boosting up once a job starts, and the boost clock itself is a ceiling higher than what the card can sustain continuously. That's a fact about silicon, not about any one cloud, but it raises a question nobody answers on a pricing page: how much of that ceiling actually survives a real workload, and how long does it take to fall?
Short benchmark runs, the kind most public numbers come from, don't answer this. A five-minute pass barely gives a card time to heat up, let alone to reach whatever thermal or power equilibrium it settles into under a real production load. The hypothesis behind this test was straightforward: a rented H100 handling continuous inference for four-plus hours would show a measurable, and probably front-loaded, gap between its rated boost clock and what it actually sustained, and that gap would show up early, not gradually erode over the whole run.
Test Setup: The GPU, the Workload, and How to Log Clock, Power, and Throughput
The setup: a single bare-metal H100 SXM5 80GB, rented specifically for direct hardware access. Bare metal matters here for a reason that has nothing to do with performance itself: it's what lets you run nvidia-smi against the physical GPU and read real clock, power, and thermal telemetry, instead of whatever a virtualization layer chooses to expose (or not) on a shared instance.
The workload is continuous LLM inference, not a synthetic stress test: a steady stream of concurrent requests against a served model, sized to keep the GPU at consistent, near-saturating utilization for the full run. No ramping load up and down; the goal is a flat, sustained floor so any clock movement can be attributed to the card's own thermal or power response, not to changes in the input.
Telemetry gets logged with nvidia-smi's CSV query mode on a short interval for the full four-plus hours:
nvidia-smi --query-gpu=timestamp,clocks.sm,clocks.max.sm,power.draw,power.limit,temperature.gpu,utilization.gpu \
--format=csv -l 2That captures SM clock against its ceiling, power draw against its cap, temperature, and utilization together, on the same timestamp, which turns out to be the whole trick to diagnosing what's actually limiting you (more on that below). For anyone reproducing this, NVIDIA's TensorRT documentation notes that locking clocks with nvidia-smi -lgc produces more deterministic, consistent measurements than letting them float, at the cost of average throughput. This test deliberately leaves clocks floating, because the point is to see what an unmanaged, real-world rental does, not what a locked benchmark configuration does.
The clock, power, and temperature figures cited through the rest of this piece come from the small set of sustained-load tests below that have actually published a full multi-hour timeline rather than a single before/after number. Run the command above against your own rented listing for the same four-plus hours and you get a timeline of your own to check against theirs.
What Sustained-Load Data Shows: Clock Speed, Power Draw, and Tokens/Sec Over Time
Run the test above for four-plus hours and the shape of the curve matters more than any single number. Clock speed doesn't creep down gradually across four hours; it moves fast early, then holds, and that shape is consistent across the sustained-load tests that have actually published a timeline rather than a single before/after number.
| Elapsed time | Reported pattern |
|---|---|
| First ~10 seconds | Full rated boost clock, before the card has had time to heat up |
| ~1 minute in | Clock falls sharply as temperature climbs toward the throttle point |
| Hour 1 onward | Clock plateaus at a lower, sustained band once thermal or power equilibrium is reached |
| Hour 2 through hour 4 | No further meaningful drift beyond the hour-one plateau |
That first row and the "~1 minute in" row are not a guess. They match a documented cloud GPU case study that logged telemetry at 10ms intervals (100 samples per second) and found full advertised performance held for only about the first 10 seconds of a sustained training run before the clock began falling. In that case, the drop was severe: an H100's clock fell from its 1,755 MHz default speed to between 450 MHz and 550 MHz within about a minute, with GPU temperature sitting around 88°C, delivering an estimated ~25% of peak performance the entire time the instance stayed billed at a flat $1,500/month. The author's own summary of that state is worth quoting directly: "the GPU is at 100% usage, but throttled between 450 MHz and 550 MHz (the GPU default speed is 1755 MHz), oh, and it's also operating at half power because of the thermal limit."
That's the number worth converting rather than just repeating: an estimated 25% of peak, billed at a flat, unchanged rate, means every unit of inference work actually delivered cost roughly four times what the sticker price implied (the full tokens/sec and cost-per-token version of that math is worked through below). It's also the entire point of running the check yourself instead of trusting a spec sheet: the gap between a card's rated clock and its sustained clock is not a fixed property of the GPU model. It's a property of that specific host's cooling and power configuration, and it can land anywhere between no meaningful gap and the roughly 75% loss documented above, depending on a listing you have no visibility into until you log it.
The First 10 Minutes vs Hour 4: Where the Curve Bends
If you only benchmark for a few minutes, the way most public GPU throughput numbers are generated, you will not see this curve bend. MLPerf's submission methodology times fixed, short-duration runs designed for controlled comparability between systems, not for surfacing what happens to a card left running for the length of an actual production shift. That's a legitimate design choice for MLPerf's purpose; it just means a leaderboard number and a four-hour production number are not the same claim, and treating them as interchangeable is where a buyer gets surprised.
The practical implication: if your own load testing tops out at ten or fifteen minutes, you are very likely measuring the tail end of the initial ramp, not the sustained state your workload will actually run at for the rest of the day. The bend in the curve happens early. Miss the first hour and you've missed the only part of the test that tells you anything.
Thermal Limit or Power Limit? How to Tell the Two Apart
A GPU sitting at a lower-than-rated clock could be thermal-limited or power-limited, and the fix for each is different, so it's worth knowing which one you're looking at. NVIDIA's TensorRT documentation lays out the method plainly: correlate temperature and power draw at the same timestamp. High temperature paired with power sitting below its cap points to thermal throttling. Power pinned flat at its ceiling with temperature still comfortable points to a power limit instead.
NVIDIA's Grace performance tuning guide draws a related distinction worth keeping in mind: throttling itself is usually an average-power-based governor response, not a single hard cutoff, and it's a separate mechanism from a critical-temperature shutdown that only fires past a much higher limit purely to protect the hardware. As the guide puts it, "the maximum frequency usually corresponds to the maximum possible performance and is higher than the frequency at which nominal (sustained) performance can be achieved." In other words, don't expect a binary "throttling / not throttling" state. Expect a governor continuously trading clock speed for staying under a power or temperature target, which shows up as the gradual settle-and-hold pattern in the table above rather than a single cliff-edge drop.
Run this diagnostic on your own listing and you'll land in one of two buckets. Temperature sitting near the roughly 85°C point TensorRT's docs cite, alongside power draw pinned at or near its cap, is the thermal-limit signature the documented case above shows: temperature near 88°C, power described as running at roughly half its normal draw. Temperature staying comfortably under that line while power still tracks below its own cap means neither limit is binding yet, and whatever clock you're seeing is closer to the card's real sustained ceiling. Same GPU generation and same physics can produce either outcome, because the limiting factor is the host's cooling and power configuration, not the chip alone.
This Isn't the Noisy-Neighbor Problem: Throttling Happens With Zero Co-Tenants
It's easy to fold this into the general "cloud GPUs are unpredictable" complaint and stop there, but it's worth being precise about which unpredictability you're looking at. Our bare metal vs. virtualized GPU performance consistency benchmark covers a different mechanism entirely: contention from sharing a physical host with other tenants, which shows up as latency variance under a hypervisor or GPU-sharing layer, and which bare metal genuinely fixes by giving you the whole card.
Clock throttling is a different failure mode, and bare metal does not fix it. It's a single-tenant phenomenon: a card with nobody else on the host, running your workload alone, still hits a thermal or power ceiling if its cooling or power cap can't keep up with sustained draw. The roughly 85°C throttle point in NVIDIA's own documentation doesn't care whether you're sharing the box. It fires on whatever GPU crosses it, virtualized or bare metal, single-tenant or multi-tenant. If you've already solved the noisy-neighbor problem by renting bare metal, that solves one variance problem and leaves this one completely untouched.
What Other Sustained-Load Tests Found (and Where Ours Agrees or Disagrees)
The evidence sits inside a small but consistent body of published sustained-load testing, and it disagrees with itself in an interesting way once you compare GPU generations. Chips and Cheese's microbenchmarking of the H100 found it occasionally dropping to as low as 1,395 MHz under sustained load, just under 80% of its boost clock, while an A100 SXM4 running comparable sustained load in the same test held a steady 1,410 MHz. That's the newer, higher-TDP card showing more clock variance than its predecessor, not less, which cuts against the assumption that a newer GPU generation is automatically the more consistent one.
An unmanaged thermal throttle on a rented instance is the same mechanism, applied by a governor you didn't configure, in a range you didn't choose, and can't predict without measuring it yourself.
Where all of this agrees is on timing: the decay, when it happens, happens fast. The documented cloud case study saw it start within a minute and settle into its throttled band shortly after. None of this data supports a slow, gradual drift over a full day; it supports a fast initial adjustment followed by a long, comparatively boring flat line, whether that flat line sits near the rated clock or, in the worst documented case, at a quarter of peak.
What This Costs You If You Don't Check It Before You Commit
The dollar consequence isn't abstract. In the documented cloud GPU case study, the instance stayed billed at a flat $1,500 a month while delivering an estimated ~25% of peak performance for the affected period, which means every unit of actual inference work done on that card cost roughly four times what the sticker rate implied, since the same monthly bill bought a quarter of the throughput. Laid out as a ratio rather than a narrative:
| Metric | Rated / expected | Documented throttled state |
|---|---|---|
| SM clock | 1,755 MHz default | ~450-550 MHz |
| Estimated share of peak performance | 100% | ~25% |
| Monthly bill | $1,500 | $1,500 (unchanged) |
| Effective cost per unit of delivered work | 1x | ~4x |
That math doesn't need a sophisticated model to matter: a buyer sizing a fleet off the advertised clock speed, rather than the sustained one, will provision for roughly four times fewer GPUs than they actually need to hit a given tokens/sec target, and only find out at the invoice or the latency dashboard, whichever comes first.
This is also where clock consistency and power draw stop being separate line items from your cost model. Our guide to AI inference power consumption and electricity costs covers the other half of this: what a GPU actually draws under load, and how that maps to a real electricity bill when you're running your own hardware. If you're renting, the equivalent question is what you're actually getting for the power the provider is already paying for. Our tokens-per-watt guide frames this as the efficiency metric that matters once you accept that peak spec-sheet throughput and delivered throughput are different numbers: a throttled card is, by definition, converting fewer of its rented watts into tokens, whatever its power bill looks like from the provider's side.
How to Check GPU Performance Consistency in the Cloud Before You Rent
Checking this yourself takes less effort than debugging it after a bad invoice or a missed SLA. A short checklist:
- Get bare-metal access if you can. It won't stop throttling, but it's what lets you read real clock, power, and thermal telemetry directly off the card instead of through an abstraction layer that may not expose it. Spheron's docs describe its bare-metal tier as delivering "bare metal performance" as part of the cost pitch against hyperscalers; that direct hardware access is also what makes a test like this one possible to run at all.
- Run your test as long as your actual job runs, not for ten minutes. The decay in the documented cases above showed up within the first minute and plateaued within the first hour; a short synthetic benchmark can easily miss the plateau entirely and report the ramp-up as the steady state.
- Log clock, power, and temperature together, on the same timestamp, with a query like the one above. Reading clock speed alone tells you throttling happened; reading it alongside power and temperature tells you why.
- Diagnose thermal vs. power using NVIDIA's own method: high temperature with power below its cap is thermal, power pinned at its cap with normal temperature is a power limit.
- Compare tokens/sec at hour one against hour four, not just against the advertised spec. That's the number your cost-per-token model should actually use.
This is one more line item worth adding to the checklist in our AI GPU buyers guide, alongside the hardware-control and hidden-cost questions it already covers. Worth being precise about what a provider's uptime figure does and doesn't promise here too: Spheron's pricing page lists a 99.9% uptime SLA and states its hourly rate covers a fully provisioned VM or bare metal instance with your selected GPU, NVMe storage, network bandwidth, and a dedicated IP with full root access, drawing on live rates aggregated across 5+ providers. An uptime SLA is a guarantee the instance stays reachable. It is not a guarantee about sustained clock speed, and neither Spheron's pricing page nor its docs publish one, which is exactly why the test above is something you run yourself on whichever listing you're renting, not something you take on faith from any provider's spec sheet.
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 15 Sep 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.
When Boost-Clock Decay Doesn't Matter
None of this is a reason to distrust every GPU listing on the market; it's a reason to know when the question is worth asking. It matters least in a few specific cases: short, bursty inference workloads with real idle time between requests, where the card never stays under sustained full load long enough to reach a thermal or power ceiling in the first place; training jobs that checkpoint and pause regularly, giving the card recovery windows a continuous inference server never gets; and workloads that are memory-bandwidth-bound rather than compute-bound, where the ceiling you're actually up against isn't the SM clock at all.
It matters most for exactly the workload this test targets: a continuous, saturating inference load running for hours at a stretch, priced and provisioned against the number on the spec sheet. If that's your job, the four-plus-hour check is worth the afternoon it takes to run.
A sustained-load check like this one is the kind of test bare-metal access exists to make possible. If your own workload runs closer to a continuous four-hour shift than a bursty afternoon, run the same
nvidia-smilog against your own listing before you commit to a fleet size.
Frequently Asked Questions
Two separate limits, and they aren't the same thing. NVIDIA's TensorRT performance documentation puts the thermal throttle point at around 85 degrees Celsius on most GPUs, past which the driver pulls clock speed down to hold temperature under that ceiling. Power throttling is a different governor: NVIDIA's Grace performance tuning guide describes it as an average-power-based response, distinct from a hard thermal shutdown that only fires past a separate critical temperature. Either one can bring a card down from its rated boost clock well before you'd notice anything from the outside; the GPU still reports 100% utilization the whole time.
Log temperature and power draw at the same timestamp with nvidia-smi and read them together, per NVIDIA's own TensorRT documentation. High temperature paired with power sitting below its cap points to thermal throttling. Power pinned flat at its ceiling with temperature well under the throttle point points to a power limit instead. Reading either metric alone will mislead you, since both produce the same symptom: a clock speed lower than the spec sheet.
No. Thermal and power throttling are properties of the physical card, its cooling, and its power cap, not of virtualization or shared tenancy. A single-tenant bare-metal instance with no other workload on the host can throttle exactly the same as a shared virtualized one, because the driver is reacting to heat and wattage, not to a hypervisor. What bare metal access does give you is the ability to run nvidia-smi directly against the physical GPU to check for yourself, instead of reading through an abstraction layer that may not expose the real clock and power telemetry at all.
Longer than a short benchmark pass. A documented cloud GPU case study found a GPU's clock beginning to fall within about a minute of sustained load and settling into a throttled range shortly after, so a test running only a few minutes can still miss the point where the curve plateaus for good. Run your own check for at least as long as your typical production job actually runs, ideally several hours, and log at a short enough interval (a few seconds or less) to see the transition, not just the start and end points.
Check the listing. Spheron aggregates live rates from 5+ providers, and its pricing page states the hourly rate covers a fully provisioned VM or bare metal instance with your selected GPU, NVMe storage, network bandwidth, and a dedicated IP with full root access. Neither tier nor cooling behavior is uniform across every listing, so confirm which instance type and tier you're renting before you assume its thermal or power-limit behavior matches another listing on the same GPU model.
