MLPerf Inference v6.1 Vera Rubin: First Benchmark Numbers (2026)

MLCommons' MLPerf Inference v6.1 round put real, peer-reviewed numbers next to NVIDIA's Vera Rubin NVL72 for the first time. The same round also carried the first trillion-parameter model MLPerf has ever scored and a new agentic-inference workload category. None of that changes what you can actually rent today. Here's what the numbers say, what they don't say, and what that means if you're deciding between a GB300 NVL72 reservation now and holding out for Vera Rubin.

TL;DR: What Do MLPerf v6.1's Vera Rubin Numbers Actually Mean?

• Submission category: That entry is Preview, not Available, so the silicon isn't a shipping, orderable product yet.
• Buy today: Spheron rents B200 at $7.20/hr and B300 at $5.40/hr on-demand as of 17 Sep 2026; GB300 and Vera Rubin stay reservation-only. Compare current pricing.

What's New in MLPerf Inference v6.1: Record Submitters, Agentic Inference, and End-to-End RAG

MLCommons is the industry consortium, with NVIDIA, AMD, Google, Intel, and most major cloud providers as members, that runs MLPerf. Submitters report performance on fixed hardware configurations, under methodology MLCommons peer-reviews before anything goes public, which is what separates an MLPerf number from a vendor's own slide deck. Here's what's actually new in this round:

• Three submission categories, and they matter: Available (hardware you can buy or rent today), Preview (near-final silicon and software, not yet orderable), and Research/Development/Internal (experimental systems not intended for sale). Vera Rubin NVL72's DeepSeek-R1 and Qwen3-VL entries both landed in Preview, confirmed by this independent MLPerf v6.1 writeup, so the system behind the headline multipliers isn't what you'd get ordering one this week.
• Agentic inference joins the suite: a new workload category scoring multi-step, tool-calling tasks instead of single-turn completions, closer to what a 2026 production LLM deployment actually runs.
• End-to-end RAG arrives too: the benchmark now scores retrieval alongside generation, instead of generation alone.
• A wider field of submitters: this round drew entries from NVIDIA, AMD, Lambda, Nebius, and other cloud and hardware vendors, including the open-division trillion-parameter run covered below.

The new agentic category enabled open-division submissions with large models that weren't possible before.

Vera Rubin NVL72 in MLPerf v6.1

NVIDIA ran its Vera Rubin NVL72 preview against multiple benchmarks, its first peer-reviewed MLPerf appearance of any kind, per the independent results breakdown.

Vera Rubin isn't the only story this round. Nebius submitted HGX B300 systems (8x Blackwell Ultra GPU nodes) and posted first-place results on DeepSeek-R1 among the providers it compared against, a reminder that Blackwell Ultra capacity you can rent today still sets competitive marks in the same round Vera Rubin is previewing in. Software improvements in NVIDIA's v6.1 submission include lower-precision KV cache, added kernel fusion, and disaggregated serving through vLLM and NVIDIA Dynamo. Some of the generational gap closes every round through serving-stack work applied to hardware you already have. For the architecture underneath both systems, see our Vera Rubin NVL72 specs guide and Rubin vs Blackwell vs Hopper comparison; our take on v6.0's results is the baseline this round builds on.

Agentic Inference and Large Models in MLPerf v6.1

The open-division entries in this round are the quieter story, and arguably the more useful one if you're not shopping for rack-scale hardware. Per the community submissions, the new agentic-inference workload category drew entries with large models, demonstrating that trillion-parameter sparse mixture-of-experts models can run reliably on smaller clusters.

If your roadmap assumes you need next-generation hardware before you can run frontier-scale MoE models in production, the open-division submissions suggest that's not a hard requirement. For the expert-parallelism and draft-head techniques that make MoE serving at this scale practical on current hardware, see our speculative decoding for MoE models guide.

What MLPerf Doesn't Tell You About Cost Per Token Today

MLPerf numbers are real and peer-reviewed, and they're also the best case your hardware will ever look. A closed-division or preview submission runs a fixed workload, on hardware a vendor's own engineers tuned for exactly that workload, with no other tenants, no cold starts, and no capacity pulled out from under it mid-run. A production deployment has all three, and each one changes the cost-per-token number MLPerf's throughput figures imply.

Start with cold starts, which MLPerf doesn't measure because its clock starts after the system is already warm. On our own 8xB200 spot fleet, bringing a node from launch to serving, with engines, weights, and compiled kernels staged on a cache volume, took 15.9 minutes for GLM-5.3, 11.5 minutes for GLM-5.3-Flash, and 11.9 minutes for DeepSeek V4.1 Flash (measured 15 and 16 October 2026). Before we added cache volumes, a node running GLM-5.3-Flash and DeepSeek V4 Flash together took about 29 minutes to reach serving. None of that shows up in a throughput-per-second MLPerf score, and all of it shows up in your hourly bill if you're paying for a node that isn't serving yet.

The first inference pass after boot has the same blind spot. On 8xB200 on 15 October 2026, GLM-5.3-Flash's first sweep ran at 34 tok/s per user at 32 users with 40-second first tokens while its linear-attention kernels compiled; warm, the same node does 129 tok/s with 0.84-second first tokens. DeepSeek V4 Flash showed the same pattern: 36 tok/s at 32 users with 8-second first tokens cold, 113 to 116 tok/s warm. A benchmark score captured after warm-up looks nothing like what your first users actually experience.

There's also a tradeoff MLPerf's single-scenario scores don't surface: how a node is sliced changes per-user speed and session count in opposite directions. Splitting an 8xB300 node into two 4-GPU copies gave lower per-user speed than running one 8-GPU copy on 8xB200: for GLM-5.3 that was 254 tok/s per user on the split B300 node versus 324 tok/s on the undivided B200 node, but the split B300 configuration served twice the concurrent agent sessions, 64 versus 32. We saw the same shape on a Kimi-family model: on 8xB200 running as one 8-GPU copy with vLLM and NVFP4 (14 October 2026), Kimi K2.7 Code held 178 tok/s for a single user and 93 tok/s at 32 concurrent users, with a 1.90-second TTFT. On 8xB300 split into two 4-GPU copies (12 October 2026), the same model ran 161 tok/s for one user and 62 tok/s at 32 users, but TTFT dropped to 0.41 seconds. Neither configuration is simply "faster." An MLPerf scenario score picks one answer, "fastest for one user" or "cheapest per session at concurrency," and reports it as the number.

Spot reclaims are the last gap, and the one that matters most if you're pricing cost per token on preemptible capacity, which is where a lot of real inference spend sits. In a live failure test on our four-node 8xB200 fleet on 16 October 2026, killing a spot node mid-stream let in-flight streams finish on the other copy after about 7 seconds with zero failures; killing a node mid-tool-call still left 16 of 16 in-flight requests with one complete, valid tool call each. A gateway hand-over during the same test dropped 0 of 832 requests. That's the difference between a benchmark's cost-per-token number and a production system's cost per completed token: the second has to account for requests that get retried, and MLPerf's controlled run never has any.

Should You Reserve GB300 Now or Wait for Vera Rubin Capacity?

The honest answer depends on when your workload needs to be live, not on the multiplier.

If you need capacity running this quarter, the Vera Rubin numbers change nothing about your decision. The hardware behind them is a preview submission, not a reservable product, with no commercial ship date firmer than "H2 2026" on pre-order pages. Reserve GB300 NVL72 now if your model plus KV cache genuinely needs rack-scale memory, or rent B200 or B300 on-demand or spot if it doesn't. On Spheron, B200 runs $7.20/hr and B300 runs $5.40/hr on-demand, per-minute billed with no long-term commitment, as of 17 Sep 2026, so you're not locked into a contract while Vera Rubin capacity opens up elsewhere. H200 is also available on-demand and spot for workloads that don't need Blackwell-class memory bandwidth. GB300 NVL72 and Vera Rubin both sit behind Spheron's reservation pages, which collect GPU count, timeline, and workload details so a request can be confirmed quickly rather than spun up instantly like an on-demand instance.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 17 Sep 2026; other providers reflect their most recent published rates and may have changed. Check current GPU pricing → for live rates.

If your workload doesn't go live until well into 2027, it's reasonable to hold a GB300 reservation request open rather than commit immediately, and watch for Vera Rubin's first available-category MLPerf submission instead of its preview numbers. That's the signal that real allocations, not engineering samples, are starting to move. Moving a workload from B200 or B300 to GB300 or Vera Rubin later carries no reservation penalty; our Vera Rubin cloud availability and cost-per-token guide and GB300 vs GB200 pricing breakdown go deeper on that transition's cost, and our vLLM on Vera Rubin NVL72 day-0 support writeup covers what's already working ahead of hardware availability.

The multiplier is real. Treat the reservation decision as separate from it.

Waiting on Vera Rubin capacity doesn't have to mean waiting on inference capacity. Rent B200 or B300 on Spheron today with per-minute billing, or reserve your spot in the GB300 NVL72 queue.

Get started on Spheron →