Migrate GPU Workloads Between Providers Without Downtime

You migrate GPU workloads between providers for one reason: the neocloud you signed with a year ago isn't the cheapest or fastest option anymore, and staying put because a move feels risky is its own cost. This isn't the guide for leaving AWS, GCP, or Azure. If you're still on a hyperscaler, our guide to migrating off AWS, GCP, and Azure covers that jump, including the egress math and containerization steps. This is about moving from one GPU-only provider to another without a training run stalling or an inference endpoint going dark.

That distinction matters more than it sounds like it should. Hyperscaler migrations are mostly about cost and data gravity. Neocloud-to-neocloud migrations are about infrastructure behavior: network tuning, driver versions, and orchestration APIs that differ just enough between two otherwise-similar GPU clouds to break a run if you don't plan the cutover. Get the checkpoint format, container packaging, and cutover pattern right, and switching providers becomes a routine operation instead of a project you keep deferring.

Why Migrating GPU Workloads Between Providers Is a Different Problem Than Leaving a Hyperscaler

The short answer: leaving a hyperscaler is mostly a data and egress problem, while moving between neoclouds is an infrastructure-behavior problem. Two providers running the same H100 SXM5 SKU can still differ enough in network fabric, driver stack, and orchestration surface that a workload built assuming one provider's defaults won't drop cleanly onto the other.

What Actually Causes Lock-In Between Neoclouds

NCCL, NVIDIA's communication library for multi-GPU and multi-node training, auto-detects network topology across PCIe, NVLink, NVSwitch, InfiniBand, and RoCE to pick the fastest collective communication path for your hardware. That's genuinely useful, but it only works across NVIDIA fabrics. NCCL is built and optimized for NVIDIA GPUs specifically, not as a cross-vendor standard, so a provider switch that also changes GPU vendor (NVIDIA to AMD, for instance) means rewriting your communication layer against RCCL instead, which is a meaningfully bigger lift than a same-vendor neocloud swap. If your migration angle covers that scenario, our NCCL tuning guide walks through the environment variables and topology settings worth re-validating on any new cluster, same vendor or not.

Beyond the network layer, three more things quietly tie a workload to a specific provider: pinned CUDA driver and toolkit versions that don't match the new provider's node images, orchestration APIs (a provider's own CLI or scheduling quirks baked into your deploy scripts), and reserved-capacity contracts that make switching a financial decision as much as a technical one. On that last point, GPU generations turn over roughly every 18-24 months, which is one reason a multi-year reservation with no exit path is a bigger bet than it looks on the term sheet. Providers that lean on vendor-financed capacity have their own incentives to keep you locked into long terms; our breakdown of NVIDIA's neocloud backstop financing is worth reading if you want the fuller picture of why some providers push multi-year commitments so hard.

The Real Objection: Downtime Risk, Not the Egress Bill

Ask a buyer why they haven't switched providers despite a better rate elsewhere, and egress fees come up less often than you'd expect. The real objection is operational: what happens to the training run in progress, or the inference endpoint currently serving production traffic, during the switch. That fear is reasonable if your only migration plan is "stop the old one, start the new one, and hope nothing breaks in between." It stops being reasonable once checkpoints, containers, and traffic routing are built to be portable from the start, which is what the rest of this guide covers.

Portable Checkpoints, Containers, and Model Weights: What to Standardize First

Before you can migrate anything without downtime, your artifacts need to load somewhere other than where they were created. That comes down to three choices: checkpoint format, container packaging, and registry location.

Use Safetensors as Your Checkpoint Format, Not Framework-Specific Pickle Files

Safetensors stopped being a Hugging Face convention and became infrastructure. On April 8, 2026, Safetensors joined the PyTorch Foundation as a contributed project, alongside PyTorch, DeepSpeed, Ray, and vLLM. That matters for portability specifically: it's no longer a format tied to one vendor's roadmap, it's governed the same way PyTorch itself is.

The technical case is just as strong. A safetensors file is a single flat file rather than a Python pickle, so loading it never executes arbitrary code, and it loads with zero-copy access, which is why Hugging Face's own documentation describes it as fast even though the safety guarantee is the headline feature. It's readable directly across PyTorch, TensorFlow, JAX, Flax, and NumPy, so a checkpoint saved on one framework's training stack doesn't require a conversion step to load somewhere else, whether that stack ends up on the same provider or a new one. If your checkpoints are still pickle files because that's what your training script defaulted to, converting them is the single highest-leverage step toward a provider-agnostic setup, ahead of anything involving containers or orchestration.

Package CUDA, Weights, and Dependencies as OCI Containers

Containerize the full stack, not just the code. A portable image bundles the CUDA runtime, your model weights (or a reference to where they load from), and every pinned dependency version, so the same image runs identically whether it's provisioned on your current provider or a new one. The detail that actually breaks migrations is driver-versus-toolkit compatibility: the container's CUDA toolkit version has to be compatible with whatever driver version the new provider's node images ship, and that compatibility window is narrower than most teams assume until a container that ran fine on one provider fails to initialize the GPU on another. Test this explicitly before you cut over any real traffic, not after.

Push to a Provider-Agnostic Registry, Not the Provider's Internal One

Pushing images to a provider's built-in registry is convenient right up until you need to pull that image somewhere else. Use Docker Hub, GHCR, or a self-hosted registry instead, so the artifact you're migrating doesn't live inside the provider you're trying to leave. This got materially easier in 2026: Kubernetes 1.36, which shipped in April 2026, graduated OCI VolumeSource to general availability, which lets model weights and configuration stored in any OCI registry mount directly as pod volumes without an init container or a custom pull step. If you're running Kubernetes across providers, that's a standard way to make model artifacts as portable as the containers that serve them; our Kubernetes GPU orchestration guide covers the broader DRA and scheduler setup this fits into. If you're not on Kubernetes at all, SkyPilot does something similar at a higher level: it treats the provider as a parameter rather than a hardcoded target, so the same job spec can launch on whichever cluster you point it at.

A Zero-Downtime Cutover Pattern for Training and Inference

Inference and training need different cutover patterns, because one is a live traffic-routing problem and the other is a distributed-state problem. Here's what each looks like in practice.

Inference: Run Both Providers in Parallel, Then Shift Traffic

Provision the new endpoint alongside the old one, not instead of it. Deploy your containerized inference stack (vLLM, TGI, or your own serving layer) on the new provider using the same portable image and checkpoint you validated earlier, and send it synthetic traffic first to warm caches and confirm the model actually loads and serves correctly. Once it's healthy, start shifting real traffic gradually with weighted DNS or your load balancer's traffic-splitting rules: 5%, then 25%, then 50%, watching latency and error rates at each step rather than flipping 100% at once. Keep the old endpoint running and reachable through the entire ramp. If something regresses at 25%, you roll back to the old provider by adjusting weights, not by scrambling to re-provision a decommissioned instance.

Training: Elastic Checkpoint Resume on a Different Cluster Topology

This is where checkpoint portability actually pays off. Say you're running a 70B-parameter fine-tuning job on 32 H100s split across 4 nodes on your current provider, and a new provider offers a better rate but only has capacity for 24 GPUs across 3 nodes right now. With a framework-locked checkpoint, that's a blocked migration. With DeepSpeed's Universal Checkpointing, it isn't: UCP is built to let a distributed run resume with a different parallelism strategy and GPU count than the one it checkpointed on, converting between configurations at a measured overhead of roughly 1.14x-1.37x rather than requiring a full retrain from scratch, and it reads checkpoints written by other training frameworks like Hugging Face Accelerate and PyTorch Lightning too. PyTorch's own Distributed Checkpoint format supports the same idea directly: you save on any number of ranks in parallel and re-shard across a different cluster topology at load time, loading into the same world size or a different one. Either way, the mid-run pause is the time it takes to provision the new nodes and re-shard the checkpoint, not a lost training run.

Validate Before You Decommission the Old Provider

Don't tear down the old provider the moment the new one is running. Compare throughput (tokens/sec or samples/sec against your baseline), loss curves for training jobs, and p95/p99 latency for inference endpoints before you call the cutover done. A model that loads and serves requests isn't automatically a model performing the same as it did on the old hardware; driver differences and interconnect topology can shift latency tails even when the median looks fine. Give the new deployment a real observation window under production load, then decommission the old provider once the numbers hold up.

Egress Costs and Contract Terms to Check Before You Commit

Between two neoclouds, the wire cost of moving data is usually the smallest part of the decision. The contract you're moving into, and the one you're leaving, matter more.

What It Actually Costs to Move Checkpoints and Datasets Off a Neocloud

Most neocloud-to-neocloud moves are cheaper than people expect. Providers like CoreWeave and Lambda Labs charge zero or near-zero egress for most workloads, against $0.05-$0.12/GB on AWS, GCP, and Azure for the same transfer. CoreWeave has taken that further: its Zero Egress Migration (0EM) program, announced in November 2025, covers the egress fees a customer's old cloud would otherwise charge, at transfers exceeding a petabyte per day, saving up to $1 million on a typical move off a hyperscaler. If a program like that exists to erase egress friction on the hyperscaler-to-neocloud leg, a neocloud-to-neocloud move, where both sides usually charge nothing to begin with, is rarely the expensive part of switching.

Spheron's own on-demand H100 pricing reflects the same no-lock-in posture: live rates aggregated across 5+ providers, no minimum rental period, and per-minute billing so you pay for exactly the time you use, whether that's a two-hour validation run on a new endpoint or a long training job. Pricing fluctuates based on GPU availability. The figures above are based on 02 Aug 2026 and may have changed. Check current GPU pricing → for live rates.

Exit Clauses and Data Export Commitments to Negotiate Before You Sign

The actual cost of leaving a provider usually isn't in the price sheet, it's in the contract you signed to get onto it. Reserved capacity is genuinely worth the discount for the right utilization pattern: six-month reservations typically run 20-30% below on-demand, one-year terms 40-50% below, and three-year terms 55-72% below, according to Compute Exchange's market benchmarking. But that discount is priced against certainty the provider gets from you, and the pricing underneath it moves. The same Compute Exchange benchmarking puts one-year H100 reserved rates at roughly $1.07 to $1.70/hr depending on provider tier, against $2.50 to $5.00/hr on-demand, and that's a market that can reprice again before your term is up.

Before you sign anywhere, get written answers on notice period, data export format, and a dated migration-assistance commitment, not a verbal assurance that you can leave if you need to. Our reservation contract negotiation guide has the full checklist, including overage pricing and step-down rights, but the short version is: never sign a multi-year commitment without confirming, in the contract itself, exactly what leaving looks like on day one of the term.

A Checklist for Migrating GPU Workloads Between Providers

Run through this before you start moving anything:

• Checkpoint format: Convert framework-specific pickle checkpoints to safetensors. Confirm your training framework supports elastic resume (UCP, PyTorch DCP, or equivalent) before you need it mid-run.
• Container packaging: Bundle CUDA runtime, weights, and pinned dependencies into a single image. Test it against the new provider's driver version before migrating real workloads.
• Registry: Push images to a provider-agnostic registry (Docker Hub, GHCR, or self-hosted), not the provider's internal one.
• Inference cutover: Run both endpoints in parallel, warm the new one, shift traffic gradually with weighted routing, and keep the old endpoint live until the ramp completes.
• Training cutover: Plan the move around a checkpoint boundary. Confirm re-sharding overhead and node/GPU count on the new provider before you start the resume.
• Validation: Compare throughput, loss curves, and p95/p99 latency against baseline before decommissioning anything on the old provider.
• Egress: Confirm the new and old provider's egress rates in writing. Most neocloud-to-neocloud moves cost close to nothing; verify rather than assume.
• Contract terms: Get notice period, data export format, and migration-assistance timelines in writing before you sign anywhere, especially on a reserved or multi-year term.

A workload built this way doesn't require a special project to move. It requires a Tuesday, a warm endpoint, and a traffic shift you can roll back if something looks wrong.

If your checkpoints, containers, and traffic routing are already portable, on-demand H100 access on Spheron can be running and validated before you commit to decommissioning anything on your current provider. Check the API reference to see how quickly you can stand up a parallel endpoint.

Check current GPU pricing →