ECMWF paid Atos more than €80 million over four years for a supercomputer that increased its compute power roughly 5x (ECMWF, 2020). GraphCast, trained on 32 TPU v4 chips for about four weeks, now beats that same institution's flagship HRES model on more than 90% of 1,380 verification targets, and produces a 10-day global forecast in under a minute on a single chip (DeepMind, 2023; Lam et al., 2023). That gap, hours of supercomputer time versus under a minute on one accelerator, is the story of AI weather modeling in 2026.
It's also a different infrastructure problem than the one most GPU cloud content covers. LLM training and serving guides dominate this space for a reason, but weather and climate models have their own data pipeline (ERA5 reanalysis, not web-scraped tokens), their own cluster math (dozens of accelerators for weeks, not thousands for months), and their own inference pattern (a scheduled cron job every few hours, not a 24/7 serving endpoint). If you're a research group evaluating whether to rent GPU capacity for a forecasting project, the frameworks built for distributed LLM training or batch inference mostly transfer, but the specifics don't. This is the rent-vs-build math for the workload as it actually runs.
Why AI Weather Models Beat Traditional NWP on Compute Cost
Traditional numerical weather prediction (NWP) solves the physical equations of the atmosphere directly: fluid dynamics, thermodynamics, radiation, on a 3D grid, stepped forward in time. It's accurate, and it's why ECMWF's HRES has been the deterministic benchmark for decades. It's also why running it requires a supercomputer, not a GPU rental.
AI weather models replace that physics simulation with a neural network trained to predict the next atmospheric state directly from the current one. The training is expensive once. The inference, the part you run every single day to get tomorrow's forecast, is cheap and fast enough to run on a single accelerator. That inversion of cost, expensive training and near-free inference, is exactly the shape of workload GPU cloud rental is built for.
The Supercomputer Economics AI Weather Models Are Disrupting
ECMWF's compute footprint gives a concrete sense of scale. In January 2020, the center signed a four-year contract worth more than €80 million with Atos for a BullSequana XH2000 system, a roughly 5x increase in ECMWF's total compute power at the time (ECMWF, 2020). Before that upgrade, ECMWF was running two Cray XC40 systems, each with more than 130,000 Intel Xeon "Broadwell" cores (ECMWF, 2016). That's the baseline capital commitment for one national weather center to run operational NWP at global scale.
GraphCast trained on 32 TPU v4 devices. Aurora's pretraining ran on 32 NVIDIA A100 GPUs for about two and a half weeks, and the team behind it explicitly notes they used "a small pool of preemptible A100 GPUs without InfiniBand" (Bodnar et al., 2024). Pangu-Weather trained each of its sub-models on 192 V100 GPUs for 16 days (Huawei, 2023). None of that requires a nine-figure procurement contract. It requires a rented cluster for a few weeks, which is the entire premise of this post: research groups that will never sign an ECMWF-scale hardware deal can now train a competitive global forecast model on hardware they lease by the hour.
Minutes vs Hours: The Inference Speedups That Matter
The accuracy gains get the headlines, but the compute-cost story is really about inference speed, because inference is what you run operationally, forecast after forecast, forever.
| Model | Inference task | Runtime | Hardware | Traditional NWP comparison |
|---|---|---|---|---|
| GraphCast | 10-day global forecast, 0.25° | Under 1 minute | 1x TPU v4 | Hours on a supercomputer with hundreds of machines |
| GenCast | 15-day ensemble forecast | ~8 minutes | 1x TPU v5 | Hours on a supercomputer consuming megawatts |
| Pangu-Weather | 24-hour global forecast | 1.4 seconds | 1x V100 | ~10,000x speedup claimed vs. traditional methods |
| NVIDIA FourCastNet 3 | 60-day rollout, 0.25°, 6-hourly | Under 4 minutes | 1x H100 | 8x faster than GenCast, 60x faster than IFS-ENS |
Sources: DeepMind GraphCast, DeepMind GenCast via AI for Cause, Huawei Pangu-Weather, NVIDIA FourCastNet 3.
Notice the hardware column. GraphCast and GenCast are Google's own models, benchmarked and served on TPU. If you're renting from a GPU cloud rather than provisioning TPU pods, the model you'll actually deploy in production is more likely NVIDIA's own Earth-2 stack, FourCastNet 3 among them, which is built and benchmarked on H100 from the start. That's worth knowing before you plan a deployment around a model's published numbers: check which accelerator family the benchmark actually ran on.
GenCast, for what it's worth, is also a strong statistical result independent of runtime: it beat ECMWF's ENS ensemble on 97.2% of 1,320 evaluation targets in a December 2024 Nature paper (AI for Cause, 2024). That's the accuracy case for AI weather models. The runtime numbers above are the cost case, and for a GPU cloud renter, the cost case is what determines your monthly bill.
Training Data and GPU Cluster Requirements for Global Forecast Models
ERA5 Reanalysis as the Shared Training Substrate
Every major model in this space trains on some slice of the same dataset: ECMWF's ERA5 reanalysis archive. The base GraphCast model trains on 39 years of ERA5 data, 1979 to 2017 (Lam et al., 2023). The operational variant, GraphCast_operational, starts from that same ERA5 pretraining and is then fine-tuned on HRES operational forecasts from 2016 through 2021, according to DeepMind's own model release (GraphCast, Google DeepMind GitHub). Pangu-Weather trained on ERA5 spanning 1979 to 2021 (Huawei, 2023). If you're planning a training run of your own, budget for the ERA5 download and preprocessing pipeline as a real line item: it's tens of terabytes of gridded reanalysis data, and staging it to fast local storage before training matters as much as the GPU count does. The GPU memory requirements for LLMs guide covers the general framework for reasoning about data staging and memory budgets that carries over here, even though the model architectures differ completely.
GPU and TPU Cluster Sizing, Compared
Here's what training actually cost each major model in accelerator-hours, where the papers disclose it:
| Model | Training hardware | Training duration | Training data |
|---|---|---|---|
| GraphCast | 32x TPU v4 | ~4 weeks (batch parallelism) | 39 years ERA5 (1979-2017) |
| Aurora (pretraining) | 32x A100 GPUs | ~2.5 weeks | 1M+ hours geophysical data |
| Pangu-Weather (per sub-model) | 192x V100 GPUs | 16 days, 100 epochs | ERA5 1979-2021 |
Sources: Lam et al., GraphCast, 2023, Bodnar et al., Aurora, 2024, Huawei Pangu-Weather, 2023.
Two things stand out here for anyone pricing out a rented cluster. First, Pangu-Weather actually trains four separate sub-models, one each for 1-hour, 3-hour, 6-hour, and 24-hour forecast steps, so the real training bill is 192 GPUs times 16 days times four, not once. Second, none of these runs are enormous by LLM-training standards. A 32-GPU cluster running for two to four weeks is a rental you can actually plan a budget around, not a hyperscaler-only proposition. For the general math on where a Slurm-scheduled, multi-week rented cluster beats an on-prem build, see the 3-year TCO breakdown for renting vs buying GPUs.
VRAM Math for Training Rollouts
Weather models don't just predict one step ahead during training. They're rolled out multiple steps forward, and the loss is computed across the whole rollout, which is what teaches the model to stay physically consistent over a multi-day forecast horizon instead of drifting after the first step. That rollout is where VRAM pressure shows up.
Microsoft's own fine-tuning documentation for Aurora is blunt about it: "to compute gradients, you will need an A100 with 80GB of memory," and that's true even with reduced precision and gradient checkpointing turned on (Microsoft, Aurora fine-tuning guide). That's not a soft recommendation, it's stated as a hard constraint for standard rollout fine-tuning. If you can't get an 80GB card, the same guide walks through FSDP sharding, Adafactor, and CPU offloading as fallbacks, all of which add engineering time you wouldn't need if you'd started with the right GPU.
This is exactly why H100 and H200 headroom matters for anyone planning longer rollout windows than the original papers used. An H100 SXM5 rental gives you the same 80GB as an A100 at meaningfully higher compute throughput, so the rollout finishes faster without changing your memory budget. If you're extending rollout length past what the reference implementation assumes, testing hourly instead of 6-hourly steps, for example, an H200 with 141GB buys the extra headroom to avoid a rewrite into a sharded training loop just to fit a longer sequence in memory.
Rent vs Build for Research Groups Without Hyperscaler-Scale Budgets
Here's the direct answer: a university atmospheric science department or an applied climate research lab cannot get a 32-chip TPU v4 pod the way DeepMind can, but it can rent a 32-node A100 or H100 cluster for the two to four weeks a training run actually needs, and walk away once the job finishes. That's the entire rent-vs-build case for this workload, and it's stronger here than for most GPU-bound research because the training runs are genuinely short.
Why University Labs Can't Get DeepMind-Scale Pods, and What a Rented Cluster Looks Like Instead
DeepMind and Google Research get first call on internal TPU pod allocation because they're training the model for a product, not a paper. A university lab or national lab research group doesn't have that internal allocation to draw on, and buying 32 GPUs outright to use them for one month a year is a bad capital allocation. Renting closes that gap: you provision a 32-node cluster, run the training job for the two to four weeks the workload actually needs, and release the capacity.
The practical setup mirrors what an HPC-background research group already knows. If your lab has run jobs on a university Slurm cluster before, running Slurm on rented GPU cloud nodes is a much smaller adjustment than learning a Kubernetes-first cloud-native stack from scratch, and it's the scheduling model most HPC centers, and Mistral in production, already use.
Multi-Node Networking Requirements for Graph Neural Weather Models
GraphCast and its successors are graph neural networks, and training them across multiple nodes means gradient synchronization at every step, same as any other distributed training job. The instinct is to assume you need InfiniBand for that. Aurora's own training run says otherwise: the team pretrained on "a small pool of preemptible A100 GPUs without InfiniBand," and noted that this limited how much scaling and ablation work they could do, not that it made training impossible (Bodnar et al., 2024).
That's a useful data point for a budget-constrained research group: InfiniBand buys you more ablation runs and faster iteration at larger node counts, but a small-to-mid cluster on standard high-speed Ethernet can train these models. Start without it, and use the multi-node GPU training without InfiniBand breakdown to see where the wall actually shows up as you scale node count, so you know when it's worth paying for the upgrade rather than assuming you need it from day one. For the deeper fabric comparison once you're evaluating whether to pay for it, the InfiniBand vs RoCE vs Spectrum-X guide covers the bandwidth and cost tradeoffs directly.
Spot vs On-Demand vs Reserved for Multi-Week Training Runs
A four-week training run is long enough that billing model matters. Three options, three different risk profiles:
- On-demand gives you a guaranteed node for the full four weeks with no interruption risk. Simplest to plan around, and worth paying for if the run is a one-off you can't afford to restart.
- Spot costs meaningfully less, but a reclaimed instance mid-run means losing progress unless you checkpoint aggressively. For a rollout-based training loop like GraphCast's or Aurora's, checkpoint after every epoch (or more often if your rollout schedule increases mid-training) so a spot interruption costs you minutes, not days.
- Reserved capacity makes sense if your lab runs this kind of training job repeatedly, a seasonal retraining cycle to fold in a new year of ERA5 data, for instance, rather than once. Locking in a rate ahead of time is worth negotiating if you know the cadence.
If your group is running these training cycles regularly enough to consider a reservation, the GPU cluster reservation contract negotiation guide covers the minimum commitments and overage terms to push back on before signing. For a one-off training run with real checkpointing discipline, spot is usually the better default: the serverless vs on-demand vs reserved GPU billing guide has the full decision framework and breakeven math if you're weighing all three at once.
Inference Cost for Running Forecasts on a Schedule
Training is a one-time cost. Inference is the number that actually shows up on your monthly bill, because you run it every forecast cycle, indefinitely.
What a Scheduled Forecast Run Actually Costs Per Day
NVIDIA reports that FourCastNet 3 computes a full 60-day, 0.25°, 6-hourly global rollout in under four minutes on a single H100 (NVIDIA, 2025). Spheron's live on-demand H100 rate is $2.54/hr as of this writing. A four-minute run costs roughly $0.17 in compute. Run that four times a day, matching the standard 00z/06z/12z/18z synoptic cycle, and you're at about $0.68/day, or roughly $20/month, for a full 60-day rollout refreshed every six hours. On spot pricing at $1.46/hr, the same schedule runs closer to $12/month.
That's the actual shape of the economics: training costs hundreds to low thousands of dollars in rented GPU time once, and operational inference costs less than a mid-tier SaaS subscription per month, indefinitely. Pangu-Weather's per-run number is even smaller in absolute terms, a 24-hour forecast in 1.4 seconds on a single V100 (Huawei, 2023), though V100 availability on current GPU clouds is limited enough that most teams will run comparably-sized models on a newer card regardless of whether they strictly need the extra headroom.
Pricing fluctuates based on GPU availability. The prices above are based on 27 Jul 2026 and may have changed. Check current GPU pricing → for live rates.
Cron-Style Operational Forecast Pipelines
An operational forecast pipeline is a batch job on a fixed schedule, not a serving endpoint waiting for requests. That's a meaningfully simpler infrastructure problem than LLM serving: no load balancer, no autoscaling group reacting to request volume, just a cron trigger that spins up an instance, pulls the latest initial conditions, runs the rollout, writes the output, and tears down.
The pattern is close to what document-processing teams already use for scheduled batch LLM jobs: provision the instance right before the run, process the batch, release the instance immediately after. The batch LLM inference on GPU cloud guide covers the queuing and provisioning patterns for this shape of workload in more depth, and most of it transfers directly, since a scheduled weather rollout and a scheduled document-summarization batch are the same infrastructure problem with different payloads.
On-Demand vs Spot for Time-Sensitive vs Research Forecast Schedules
The billing decision here comes down to one question: does a missed cycle cost you something real? An operational pipeline feeding a downstream system that people or automated processes act on, aviation routing, agricultural irrigation timing, a disaster-response dashboard, needs on-demand. A reclaimed spot instance mid-run means a forecast that simply doesn't arrive, and for a time-sensitive downstream consumer, a missing 06z cycle is worse than a slightly more expensive one.
A research pipeline generating forecasts for model comparison, backtesting, or a paper's evaluation set has more slack. If a run gets reclaimed, you requeue it and it completes an hour later; nobody downstream was waiting on that specific cycle. That's a case where spot's discount is close to free money.
WindBorne Systems is a useful real-world data point here too, though its edge is a different one than raw inference speed. The startup runs its WeatherMesh-6 model with hourly updates at 3km resolution over the continental US, versus the 6-hour cadence typical of traditional forecasts, and claims 5-day-out accuracy comparable to a traditional forecast one day out (TechCrunch, 2026). What backs that isn't a bigger model, it's WindBorne's own network of roughly 400 weather balloons feeding real-time sensor data into training and initialization. As CEO John Dean put it: "I don't understand, personally, the business model of being an AI based weather company without a dataset advantage" (TechCrunch, 2026). The inference compute is cheap for everyone in this space now. What differentiates one AI weather model from another is increasingly the input data, not the accelerator it runs on.
For power-and-electricity-adjacent cost questions that come up once you're running dozens of scheduled forecast cycles a day across a fleet of GPUs, the AI inference power consumption and electricity cost guide breaks down the TDP and cooling variables that a rented GPU cloud's per-hour price already absorbs for you.
If your lab is scoping a GraphCast-style training run or standing up a scheduled FourCastNet-style forecast pipeline, an on-demand or spot A100, H100, or H200 cluster gets you there without a multi-year hardware contract.
Frequently Asked Questions
GraphCast trained in about four weeks on 32 Google Cloud TPU v4 devices using batch parallelism, on 39 years of ERA5 reanalysis data. A research group without TPU pod access can approximate this with a rented multi-node A100 or H100 cluster running the open-source JAX implementation, though wall-clock time will vary with GPU count and interconnect.
Not always. Microsoft's Aurora team pretrained on a small pool of preemptible A100 GPUs without InfiniBand, according to the paper. Graph neural weather models exchange gradients at each step like any distributed training job, so InfiniBand still helps at larger node counts, but small-to-mid research clusters can start on standard Ethernet and add IB only if scaling tests show a network bottleneck.
Microsoft's own fine-tuning guide for Aurora states you need an A100 with 80GB of memory to compute gradients, even with reduced precision and gradient checkpointing enabled. That is the practical floor for research-scale rollout fine-tuning; H100 80GB is a drop-in equivalent, and H200's 141GB gives headroom for longer rollout windows without hitting the same wall.
It depends on the forecast's purpose. A scheduled research forecast that runs a few times a day and can tolerate a delayed cycle is a good fit for spot pricing. An operational pipeline feeding a time-sensitive downstream system (aviation, agriculture alerts, disaster response) should run on-demand, since a reclaimed spot instance mid-cycle means a missed forecast window.