CUDA News Today: NVIDIA Toolkit, AMD ROCm, and AI Framework Releases (2026)

<!-- WEEKLY UPDATE PROTOCOL:

1. Prepend a new "This Week" section (5 news items with source links) above the previous week's entry.
2. Update the date: frontmatter field to the current Monday's date.
3. Update any tracker tables (CUDA, ROCm, Framework, Driver) if a new version shipped.
4. Append the previous week's top story as a one-line entry in the Archive section.
5. Commit and push - the Vercel build updates the static page and lastmod in sitemap.

-->

This is a weekly digest of CUDA toolkit releases, AMD ROCm updates, and AI framework compatibility changes that affect GPU cloud workloads. It updates every Monday with the latest releases, version tables, and a "What this means for Spheron users" section tying each update to real deployment decisions. For deeper background on the CUDA vs ROCm divide, see the ROCm vs CUDA comparison. For a deep dive into CUDA 13's new programming model, see the CUDA 13 Tile programming guide.

This Week in CUDA and GPU Compute (Week of 19 May 2026)

1. CUDA 13.2 stable released

May 15, 2026 - CUDA Toolkit 13.2 Release Notes

CUDA 13.2 ships the cuTile Python DSL as a stable feature after its experimental debut in 13.1. It also extends the Cooperative Groups API with new grid_group synchronization primitives that simplify multi-block reductions. CUDA 13.x dropped support for Maxwell, Pascal, and Volta but continues to support Turing, Ampere, Ada, Hopper (H100/H200, compute capability 9.0), and Blackwell. H100 and H200 users can now run CUDA 13.x without pinning to 12.8.

2. ROCm 7.2.3 patch ships vLLM profiling improvements for MI300X and MI350X

May 2026 - ROCm Release Notes

AMD pushed ROCm 7.2.3 with improved profiling stability for vLLM workloads traced with PyTorch torch.profiler, reducing large idle gaps between GPU kernels. The release also merges multiple gather operations from similar embedding tables into batched operations, cutting kernel launches and memory traffic for embedding-heavy inference. ROCm 7.2 is the current stable branch for MI300X and MI350X production deployments. For a comparison of AMD and NVIDIA software stacks, see the ROCm vs CUDA comparison.

3. PyTorch 2.12 stable released

May 13, 2026 - PyTorch 2.12 Release Notes

PyTorch 2.12 is the current stable release. Key additions include batched linalg.eigh on CUDA that is up to 100x faster for symmetric eigendecompositions and continued torch.compile improvements for CUDA 13.x kernels. CUDA 12.8 and 13.x wheel variants are both available. For production deployments with CUDA Graphs and persistent compilation caching, the torch.compile and CUDA Graphs guide covers the current recommendations.

4. vLLM 0.21.0 released with KV offload and Hybrid Memory Allocator

May 15, 2026 - vLLM v0.21.0 Release Notes

vLLM 0.21.0 ships KV offload integration with the Hybrid Memory Allocator (HMA), enabling larger effective context windows by tiering KV cache between GPU and CPU memory. It also adds speculative decoding support for reasoning models with thinking budget constraints and the TOKENSPEED_MLA backend on Blackwell for DeepSeek-R1 and Kimi-K25. Eagle3 speculative decoding (added in v0.20.0) continues to be available. See the vLLM production deployment guide for current flag recommendations.

5. OpenAI Triton (triton-lang) 3.7.0 released

May 7, 2026 - OpenAI Triton (triton-lang) 3.7.0 Release Notes

OpenAI Triton (triton-lang) 3.7.0 ships with continued improvements to both the CUDA and ROCm backends, improving performance for flash attention and linear attention patterns on CDNA3 and Blackwell hardware. The block pointer API is stable across recent releases. Note: this is the kernel language OpenAI Triton (triton-lang), not NVIDIA Triton Inference Server. For GPU kernel development context, see the Triton kernel development guide.

What This Means for Spheron Users (Week of 19 May 2026)

CUDA 13.2 supports Turing, Ampere, Ada, Hopper, and Blackwell architectures. H100 and H200 users can now run CUDA 13.x without pinning to CUDA 12.8. For teams exploring B300 SXM6, spot instances are currently available at $2.45/hr per GPU, making CUDA 13.x workloads low-cost to validate.

When deploying on Spheron, select an NGC base container matching your target CUDA version (e.g., nvcr.io/nvidia/pytorch:24.08-py3 for CUDA 12.8 or a newer tag for CUDA 13.x). The container handles the toolkit; the bare-metal driver is pre-installed at the host level. For automating driver and toolkit setup at instance launch, see the Spheron startup scripts guide.

H100 SXM5 instances start at $1.66/hr spot or $2.64/hr on-demand, with SSH root access included. B200 SXM6 starts at $3.77/hr spot or $7.21/hr on-demand for teams that need Blackwell.

CUDA Toolkit Release Tracker

CUDA 13.x dropped support for Maxwell, Pascal, and Volta (pre-Turing). Turing, Ampere, Ada, Hopper (H100/H200), and Blackwell are all supported. See the CUDA 13 Tile programming guide for details on the new programming model.

Source: CUDA Toolkit Release Notes

AMD ROCm Release Tracker

ROCm 7.2 is the current stable series. MI350X and MI355X have been fully supported since ROCm 7.0.0. MI400 remains in developer preview. For a full comparison of AMD and NVIDIA software stacks and when to choose each, see the ROCm vs CUDA comparison.

Source: ROCm GitHub Releases

Framework Compatibility Matrix

For PyTorch 2.6 in production with CUDA Graphs and persistent compilation caching, see the torch.compile and CUDA Graphs guide.

NVIDIA Datacenter Driver Branches

Use R580 for production LLM inference workloads on H100, H200, A100, B200, and B300. It is the current Long Term Support branch with an EOL date of August 2028, making it the stable choice for fleet deployments. R570 (production branch) reaches end of life in June 2026. R575 was a short-term support branch and is already EOL. Both R570 and R575 users should migrate to R580 before the R570 EOL date, and R575 users should treat the migration as urgent since that branch is no longer receiving security or stability fixes. LTS branches are ideal when you want a fixed driver ABI across a large fleet.

Source: NVIDIA Driver Documentation

AMD Instinct Driver Updates

Source: AMD ROCm GitHub and AMD ROCm Documentation

Library and Runtime Updates

cuDNN

cuDNN 9.7 (released April 2026) is the current stable release. It adds FP8 convolution support on Blackwell and extends the graph API with fused attention patterns for GQA (Grouped-Query Attention) layouts. The graph API is now the primary path for custom attention implementations; the legacy C API is deprecated but not removed. Compatibility: requires CUDA 12.4 or later.

cuBLAS and cuSPARSE

cuBLAS 12.8.4 (shipped with CUDA 12.8) added FP8 GEMM variants tuned for Blackwell's new tensor core layouts. The API interface is stable; no breaking changes from 12.6. cuSPARSE 12.5 added structured sparsity patterns for 2:4 pruning on Hopper and Blackwell without requiring a custom kernel.

NCCL and RCCL

NCCL 2.23 (current stable) improves All-Reduce bandwidth on 400G InfiniBand fabrics and adds adaptive tree algorithms that reduce latency for small tensors in pipeline-parallel setups. RCCL 2.21 (AMD's counterpart) brings NCCL API parity for the ring and tree collectives used by PyTorch DDP and DeepSpeed.

For tuning NCCL for multi-GPU training jobs, see the NCCL tuning guide.

TensorRT

TensorRT 10.9 is current. It ships FP4 sparsity on Blackwell GB200 and extends the FP8 quantization path to cover KV cache compression without a custom plugin. The deprecated TF32 fallback for FP16 overflow is removed in 10.9; if your network had silent TF32 promotion, you will see explicit errors now. For production LLM serving setup with TensorRT, see the TensorRT-LLM deployment guide.

vLLM

vLLM 0.21.0 (current stable as of this week) ships KV offload integration with the Hybrid Memory Allocator, speculative decoding support for reasoning models with thinking budget constraints, and the TOKENSPEED_MLA backend on Blackwell. Eagle3 speculative decoding and EPLB load balancing for MoE models (added in v0.20.0) are fully available. The --enable-prefix-caching flag is on by default for multi-turn workloads. See the vLLM production deployment guide for current flag recommendations.

Compiler and Kernel-Level Releases

OpenAI Triton (triton-lang)

OpenAI Triton (triton-lang) 3.7.0 is current stable. The block pointer API is stable across recent releases. Both the CUDA and ROCm backends have received continued improvements for flash attention and linear attention patterns on Blackwell and CDNA3 hardware. For writing production custom kernels with OpenAI Triton (triton-lang), see the Triton kernel development guide.

CUTLASS

CUTLASS 3.9 (April 2026) adds new tile shapes for Blackwell's FP8 MMA instruction set and improves the Python GEMM template API. The cute layout algebra is now the primary API path; the legacy CUTLASS 2.x API is in maintenance mode.

ThunderKittens

ThunderKittens (HazyResearch) is production-ready for H100 SXM5 targets and has experimental Ada (RTX 4090) support. Blackwell targeting is in active development. If your use case involves writing custom attention kernels for H100, ThunderKittens provides a higher-level abstraction than raw CUDA PTX.

FlashAttention

FlashAttention 3 is stable on Hopper (H100/H200) and is the recommended kernel for production transformer inference. FlashAttention 4 targeting Blackwell (B200/B300) is in beta. For a detailed guide on deploying FlashAttention 4 on Blackwell instances, see the FlashAttention 4 guide.

AI Agent and Inference Framework News

SGLang

SGLang 0.4.x is current stable. Notable additions include a new RadixAttention scheduler with better KV cache utilization for shared prefix scenarios and improved MoE expert routing. For production SGLang setup on Spheron instances, see the SGLang production setup guide.

vLLM

vLLM 0.21.0 is current (see Library and Runtime Updates above). The key deployment decision for new vLLM setups is whether to use the AsyncLLMEngine (best for per-request latency) or the V1 engine (best for throughput at high concurrency). See the vLLM production deployment guide for when to use each.

TensorRT-LLM

TensorRT-LLM 0.17 added FP4 inference for Blackwell GB200, which is the first production-grade sub-FP8 inference path for transformer models. It also ships improved in-flight batching for speculative decoding. For setup and production configuration, see the TensorRT-LLM deployment guide.

NVIDIA Dynamo

NVIDIA Dynamo 0.3 (May 2026) ships disaggregated prefill and decode routing as a stable feature. It targets clusters with 8+ nodes and works with both TensorRT-LLM and vLLM backends. For a full setup walkthrough, see the NVIDIA Dynamo disaggregated inference guide.

llm-d

llm-d 0.2 (April 2026) is the Kubernetes-native inference framework for disaggregated LLM serving. It ships a custom scheduler plugin and a Kubernetes operator for managing disaggregated prefill/decode pods. For a production setup on Kubernetes with Spheron instances, see the llm-d on Kubernetes guide.

GPU Pricing for CUDA Workloads (as of 20 May 2026)

The following pricing is from Spheron's live GPU marketplace. Use these figures to estimate compute costs when planning a CUDA or ROCm upgrade cycle.

B300 spot instances at $2.45/hr are currently the cheapest way to test CUDA 13.x workloads on Blackwell. H100 on Spheron starts at $1.66/hr spot (on-demand from $2.64/hr) for CUDA 13.x production deployments. A100 instances on Spheron at $1.15/hr on-demand are a solid choice for batch jobs.

Pricing fluctuates based on GPU availability. The prices above are based on 20 May 2026 and may have changed. Check current GPU pricing for live rates.

Archive: Past Weekly Digests

1. [Week of 12 May 2026] - vLLM 0.20.0 released with Eagle3 speculative decoding and EPLB load balancing for MoE models.
2. [Week of 05 May 2026] - ROCm 7.2.2 maintenance release shipped with vLLM profiling stability fixes; PyTorch 2.13 dev wheels landed on PyPI.
3. [Week of 28 Apr 2026] - NCCL 2.23 released with improved bandwidth on 400G InfiniBand; TensorRT 10.9 added FP4 for Blackwell.

<!-- Add new weekly entries above this line. Move entries below #20 to a separate archive page when this list grows beyond 20 items. -->

CUDA and ROCm releases move fast. The simplest way to stay current is to run each new toolkit version on bare-metal instances where you control the driver stack and can verify behavior against your own kernels before pushing to production. Spheron gives you SSH root access and the ability to swap base images on every deployment, with no GPU sharing across tenants.

Rent H100 on Spheron | B200 GPU cloud | H200 GPU rental | Deploy now