Instead of provisioning a virtual machine and SSHing in to install a stack yourself, you point a public Docker image at a GPU and it runs, the same motion as docker run against hardware you don't own. This is a step-by-step walkthrough of exactly how, written three days after launch.
TL;DR: How to Run a Docker Container on a GPU with Spheron Containers
Spheron Containers, launched 30 October 2026, runs any public Docker image on a GPU or CPU with no virtual machine to provision, per Spheron's own docs.
- What launched: GPU and CPU container deploys across 28 GPU types, from RTX 2000 Ada and A40 up to H100, H200, B200, and B300, in 1, 2, 4, or 8-GPU sizes, per Spheron's 30 Oct 2026 changelog.
- Templates: GPU templates ship for PyTorch, CUDA, vLLM, SGLang, Ollama, Open WebUI, TensorFlow, and Jupyter; CPU templates cover Ubuntu, Debian, Python, Node.js, Go, Rust, and Jupyter.
- Storage: every container gets a 100 GB container disk plus a 500 GB /workspace volume by default; a standalone volume can be sized from 10 to 4096 GB and shared across several containers at once.
- Billing: stopping a GPU container keeps its /workspace volume and bills only storage; CPU containers have no Stop state, only restart or terminate.
- Bring your own image: any public image from Docker Hub, GHCR, or Quay, with up to 50 environment variables and up to 9 exposed ports, each with its own public address. See Spheron's H100 GPU rental page for a live on-demand rate.
What Spheron Containers Launched on 30 October 2026
Spheron's own concept page puts it in one line: Containers runs "a public Docker image on a GPU or a CPU, without a virtual machine to look after." That's the whole shift. A Spheron GPU instance before this was still a VM underneath, one you provisioned, booted, and configured by hand or by script. A container deploy skips that layer: you point at an image, the platform schedules it onto GPU or CPU hardware, and the entrypoint that image already defines just runs.
A standalone volume can be resized anywhere from 10 GB to 4096 GB (4 TB) and mounted at /workspace, where more than one container can share it at the same time, a different product shape from the 100 GB container disk bundled into every deploy by default.
This is a new entry point, not a rename of the existing instance flow. Spheron's Stop and Start feature, shipped in September 2026, already let a full GPU instance pause and resume without losing its disk. Containers brings that same idea down to the unit a lot of AI workloads actually ship in: a single image, not a whole provisioned machine.
How to Run a Docker Container on a GPU: Step-by-Step Deploy
- Choose GPU or CPU. Decide whether the workload needs GPU hardware at all; a CPU container is cheaper for anything that doesn't touch a model.
- Pick the GPU type and count. Select from the 28 supported GPU types and a size of 1, 2, 4, or 8 GPUs, matched to how much VRAM and parallelism the workload needs.
- Pick a template or bring your own image. Use a launch template (PyTorch, CUDA, vLLM, SGLang, Ollama, Open WebUI, TensorFlow, Jupyter on GPU; Ubuntu, Debian, Python, Node.js, Go, Rust, Jupyter on CPU), or point at any public image on Docker Hub, GHCR, or Quay.
- Set environment variables and ports. Configure up to 50 environment variables and expose up to 9 ports, each with its own public address, no manual reverse proxy setup required.
- Size the /workspace volume if the default doesn't fit. The default is a 100 GB container disk plus a 500 GB /workspace volume; resize a standalone volume anywhere from 10 GB to 4096 GB if needed.
- Deploy. The image is pulled and scheduled onto the selected GPU or CPU hardware, and SSH is set up automatically with your key if the image runs as root.
That's the full path from a public image to a running GPU container, with no VM-provisioning step in between.
Templates for PyTorch, CUDA, vLLM, SGLang, Ollama and More, and When to Bring Your Own Image
The launch templates map onto different jobs. PyTorch and CUDA suit a training script or a custom inference loop you're writing yourself, where you want the framework and drivers ready but nothing else opinionated about serving. vLLM and SGLang are for production-style LLM serving: continuous batching, PagedAttention or RadixAttention, and an OpenAI-compatible endpoint out of the box, covered in more depth in Spheron's vLLM production deployment guide and SGLang production deployment guide. Ollama and Open WebUI are the fastest path to a chat interface in front of an open model, TensorFlow covers jobs still built on that framework, and Jupyter, on both GPU and CPU, is for interactive exploration rather than a long-running service.
A template is a starting point, not a ceiling. The moment a workload needs a dependency or a serving stack a template doesn't carry, bring your own image instead. Spheron's NVIDIA NIM self-host guide is the clearest existing example on the blog of that path: NIM ships as a prebuilt container image deployed onto GPU hardware rather than configured from scratch, precisely the custom-image case Containers now handles natively instead of through a hand-provisioned VM.
Serverless GPU vs On-Demand vs Reserved: Where a Container Deploy Fits
Serverless, on-demand, and reserved are three answers to the same question: what do you pay for when the GPU isn't actively working? Each trades startup latency against idle cost differently, and a container deploy on Spheron lands on the on-demand (or spot) side, not the serverless side.
Serverless GPU means the platform decides when hardware exists at all. A request arrives, a container starts, and billing runs only while it's actively loading or processing; once it scales back to zero, billing stops with it. Modal's own pricing page prices H100 at $0.001097 per second, roughly $3.95/hr, billed only while a container is processing input or loading, with nothing charged once it scales to zero. That's excellent for spiky, unpredictable traffic, and it costs a cold start, often tens of seconds to minutes, on every scale-up from zero. How long that cold start actually runs depends on what's loading, not just whether a container exists at all: in our own testing on 8xB200 spot nodes with engines, weights, and compiled kernels cached on a volume, bringing a large model up from a stopped state measured 15.9 minutes for GLM-5.3, 11.5 minutes for GLM-5.3-Flash, and 11.9 minutes for DeepSeek V4.1 Flash as of October 2026, with getting the VM itself accounting for only 2 to 3.5 minutes of each figure and the rest going to engine install, weight loading, and warm-up.
On-demand flips the trade. You pay by the hour for a GPU that stays allocated and warm the whole time, whether or not it's actively processing at any given second. There's no cold start between requests because there's no scaling to zero between them. Runpod's own pricing page lists H100 SXM Secure Cloud (on-demand) at $3.99/hr and H100 PCIe at $2.89/hr, against $5.39/hr for the same H100 on Runpod Serverless, a direct illustration of the premium a provider charges for staying always-warm. A Spheron Containers deploy sits here: the container runs continuously once deployed, billed hourly, until you stop or terminate it.
Reserved goes further than on-demand by trading flexibility for a lower rate: commit a GPU, or a cluster of them, for a fixed term in exchange for pricing below the hourly on-demand rate. It fits once a workload's GPU-hours are predictable enough for a multi-week or multi-month commitment, and it's the wrong fit for anything still in the experimentation phase.
| Model | Billed while idle? | Cold start | Best fit |
|---|---|---|---|
| Serverless | No, scales to zero | Yes, on every scale-up | Bursty, unpredictable traffic |
| On-demand | Yes, while allocated | No, stays warm | Steady or long-running workloads |
| Reserved | Yes, for the committed term | No, stays warm | Predictable, sustained GPU-hours |
| Spot | Yes, while running, can be reclaimed | No while running | Fault-tolerant, checkpointed workloads |
A container deploy on Spheron is an on-demand (or spot) GPU wearing a Docker-native front door instead of a VM-provisioning one. That's different from Modal or a serverless inference endpoint: a Spheron container stays warm and billed hourly for as long as it runs, rather than scaling to zero and re-paying a cold start between calls. If a workload is genuinely spiky request-by-request traffic, a true serverless platform like Modal is the better fit on paper. If it's a service, an agent, or a training job that needs to stay up and responsive, on-demand, what a container deploy gives you, is the right tier, and bringing a Docker image to it is now faster than provisioning a VM first.
Pricing fluctuates based on GPU availability. The Modal and Runpod figures above were checked against their own pricing pages on 5 November 2026 and may have moved since. Check current GPU pricing for live Spheron rates today.
Stop/Start and Persistent /workspace Volumes Across 28 GPU Types
Stop/Start on containers works the way it works on Spheron's full GPU instances, applied to a smaller unit. Stopping a GPU container releases the GPU and drops billing to storage only, while the /workspace volume, and everything written to it, stays exactly where it was. Starting again picks that same volume back up, so a model already downloaded or a checkpoint mid-run doesn't need to be re-fetched or rebuilt.
The default allocation, 100 GB container disk plus 500 GB /workspace volume, covers most single-model workloads without resizing. Workloads that need more room, a larger dataset, multiple checkpoints, or a shared cache across several containers, can provision a standalone volume from 10 GB up to 4096 GB and mount it at /workspace, where several containers can read and write it concurrently. That lets, say, a training container and a serving container on different GPU types both see the same checkpoint directory without copying it between them.
CPU containers don't get this pause option. There's no Stop state for a CPU container at all, only restart or terminate, worth knowing before building a workflow around pausing a CPU container the way you would a GPU one.
The Stop behavior itself doesn't vary across the 28 supported GPU types, only the hourly rate you're pausing from does. A single H100 container and an 8xB200 container both keep their /workspace volume and drop to storage-only billing on Stop; the dollar amount that stops accruing is just much larger on the 8-GPU deploy.
Ports, SSH Access, and Custom Registries for Private Images
Bringing your own image is where Containers stops being a template picker and becomes a general-purpose way to run anything on a GPU. Spheron accepts any public image from Docker Hub, GHCR, Quay, or another public registry. Per the changelog, a custom deploy supports up to 50 environment variables, enough to configure a serving stack's model path or API keys without editing the image itself, and up to 9 exposed ports, each given its own public address rather than sharing one, useful for a container that serves an API on one port and a metrics endpoint on another. SSH is handled the same way as on Spheron's full instances: if the image runs as root, Spheron sets up SSH automatically with your key, so you can shell in to debug a custom image without baking an SSH server into it yourself.
The honest limit is registry scope. What's documented is support for public registry images; a team that needs a private or authenticated registry image, the kind gated behind a login for a proprietary model or an internal base image, isn't clearly served by what's published so far. That's worth confirming directly against current docs before planning a migration around it, since this is also a three-day-old feature without a long track record yet.
Common Mistakes When Deploying a Docker Image on a GPU
- Assuming a container deploy behaves like serverless. A container on Spheron stays warm and billed hourly until stopped or terminated; it doesn't scale to zero between requests the way Modal does. Budget it as an on-demand GPU, not a pay-per-inference endpoint.
- Treating the bundled container disk as permanent. The 100 GB container disk is part of the container's own lifecycle; /workspace is the piece that survives a Stop, so don't put anything you need to keep outside it.
- Forgetting that CPU containers can't be paused. There's no Stop state on the CPU side, only restart or terminate, so a CPU workload left running accrues full charges the whole time.
- Exposing every port by default instead of only what's needed. Each of the up to 9 ports gets its own public address; exposing a debug or admin port you don't need reachable widens the attack surface for no benefit.
- Picking a GPU size without checking what the model needs. An 8-GPU container costs roughly eight times a 1-GPU container of the same type; size to VRAM footprint and parallelism, not to "more GPUs are always safer."
Spot capacity, on Spheron or anywhere else, adds one more thing worth planning for regardless of whether you're deploying a VM or a container: reclaims happen without warning. Build for the reclaim before it happens: keep two copies of a model on different nodes, spread across regions or providers, and set up automatic recovery, rather than relying on a single spot node staying up for the length of a run.
From Docker Image to Running GPU Container
Spheron Containers turns "rent a GPU" into something closer to docker run than to provisioning a VM: pick a GPU, pick a template or a public image, set your ports and environment variables, and deploy, with Stop/Start and a persistent /workspace volume handling the parts that used to mean babysitting a machine by hand. It launched three days before this post, covers 28 GPU types from RTX 2000 Ada through B300, and the bring-your-own-image path means it isn't limited to the eight launch templates for long.
Run a Docker container on a GPU without provisioning a VM first. Spheron Containers covers 28 GPU types from RTX 2000 Ada to B300, with launch templates for PyTorch, CUDA, vLLM, SGLang, Ollama, and more, or bring your own public image.
Frequently Asked Questions
Pick a provider that runs containers directly on GPU hardware, pick a GPU and a template or your own public image, set any environment variables and ports the image needs, and deploy. On Spheron Containers, that whole flow happens in the dashboard: choose a GPU type and count, pick a GPU template (PyTorch, CUDA, vLLM, SGLang, Ollama, Open WebUI, TensorFlow, or Jupyter) or paste a custom image from Docker Hub, GHCR, or Quay, then deploy. There's no VM to provision or SSH into first.
Serverless GPU bills per second only while a request is actively processing and scales to zero between calls, at the cost of a cold start on every spin-up. On-demand bills by the hour for a GPU that stays allocated and warm the whole time, with no cold start on the next request. Reserved locks in a GPU, or a cluster of them, for a committed term at a lower rate than on-demand, in exchange for giving up the ability to walk away hourly. A container deploy on Spheron runs on-demand or spot pricing: the container is live and warm until you stop or terminate it, not billed per inference call.
Yes, on Spheron Containers. Stopping a GPU container keeps its /workspace volume in place and drops billing to storage only, the same cost-pause model Spheron already ships for full GPU instances. CPU containers work differently: there's no Stop state for them at all, only restart or terminate, so there's no storage-only pause option on the CPU side.
Yes. Spheron Containers accepts any public image from Docker Hub, GHCR, Quay, or another public registry, with up to 50 environment variables and up to 9 exposed ports, each getting its own public address. If the image runs as root, Spheron sets up SSH automatically with your key. The published support is for public registry images; a private or authenticated registry isn't clearly covered by what's documented yet.
