GPU Cloud Egress Costs: The Hidden AI Bandwidth Bill (2026)

Your GPU bill shows $12,000/month for H100 compute. Your actual AWS bill is $14,200. The $2,200 difference is egress, and it appears nowhere in the GPU rental comparison you ran before choosing AWS.

This is the most predictable surprise in AI infrastructure billing. Hyperscalers publish GPU compute rates prominently and bury data transfer pricing in a separate, tiered pricing page. For text-only inference at low volume, egress barely registers. For multimodal pipelines, high-frequency streaming APIs, or workloads that pull model weights from S3 on every restart, the bandwidth bill can exceed 20% of total monthly spend.

This guide breaks down where every egress dollar goes, shows you the math for common AI workload patterns, and covers both quick optimizations and a full migration path to zero-egress infrastructure.

Egress Pricing: What the Hyperscalers Actually Charge

AWS, GCP, and Azure all use tiered egress pricing. The first tier covers most small and mid-size AI deployments, but the rate doesn't drop enough to matter at production scale.

Verify these rates against current provider pricing pages before committing. Hyperscaler egress pricing does change. AWS revised P5 compute pricing in mid-2025, but egress tiers have remained more stable.

There's also a second cost category that many engineers overlook: cross-AZ data transfer. When your inference pods run in one availability zone and your load balancer or API gateway sits in another, AWS charges $0.01/GB in each direction. For a high-traffic endpoint with a standard multi-AZ deployment, this adds up independently of internet egress. Keep your inference GPU, load balancer, and downstream services in the same AZ where possible to eliminate this line item entirely.

For teams dealing with other unexpected AWS costs, egress is typically the largest hidden expense after reserved instance complexity and cross-service data transfer charges.

The Token Streaming Math

Text-based LLM endpoints don't generate as much egress as most engineers assume. Understanding the exact byte budget helps you prioritize which workloads to optimize first.

Bytes per token: UTF-8 encodes English text at roughly 3-5 bytes per token using a standard BPE tokenizer. For a typical mix of English prose and code, 4 bytes per token is a solid working estimate. Multilingual output skews higher (Chinese and Japanese characters are 3 bytes each but map to fewer tokens). Code-heavy outputs with long identifiers run lower.

Streaming overhead: Each SSE frame includes HTTP metadata: a data: prefix, JSON wrapper, newline delimiters, and chunk framing. Per-token overhead adds roughly 20-40 bytes per streamed chunk depending on your response envelope. For a small-batch model that streams one token per SSE event, this multiplier is significant. A 1.5x to 2x effective multiplier over raw token bytes is realistic for streaming endpoints.

Pure text at these volumes is manageable. The AI inference cost economics analysis covers the full unit economics picture, including compute and batching effects. Egress is rarely the dominant cost for text-only workloads.

Where it shifts: when response payloads grow. An LLM that appends retrieved document excerpts to its response, returns base64-encoded images, or serves audio transcripts in the response body moves the egress needle fast. The same math applies, but with KB-per-response instead of bytes-per-response.

High-Egress AI Workloads

LLM Token Streaming Endpoints

Standard streaming text inference is the lowest-egress AI pattern. The numbers above show that even a billion tokens per day generates only 180 GB/month of egress before overhead. The concern here is TCP framing at very high request-per-second rates. At 10,000 streaming requests/second, the cumulative overhead from connection establishment and SSE framing can push effective bytes-per-token above the 1.5x multiplier. HTTP/2 multiplexing reduces this significantly.

Video Generation Pipelines

Video generation creates the most egress of any AI workload by a wide margin. A single 720p, 5-second clip generated by a diffusion model runs 3-8 MB depending on codec and quality settings. At 1080p, multiply by 4. A pipeline generating 10,000 clips per day at 5 MB each produces 50 GB/day of outbound traffic, or 1.5 TB/month. That's $135/month in egress on AWS, on top of the significant GPU compute costs for generation. Serving video outputs through a CDN with edge caching eliminates most of this, since generated clips are static artifacts.

Audio Synthesis APIs

Text-to-speech output runs 20-200 KB per second of audio, depending on codec. WAV files at 16kHz/16-bit/mono: about 32 KB/second. MP3 at 128kbps: 16 KB/second. For a TTS API serving 1 million 30-second responses per day, that's 960 GB/day in WAV format or 480 GB/day at 128kbps MP3. Encoding output as Opus at 48kbps (standard for voice-quality audio) cuts this to 180 GB/day. Codec selection matters more for egress than for compute cost.

Multimodal RAG

RAG pipelines that retrieve and return images compound the egress cost fast. The typical pattern: retrieve 3-5 document chunks, where some are image pages from a PDF, and embed thumbnails or base64-encoded images in the JSON response. A 300x300px JPEG thumbnail is roughly 15-25 KB. If your average RAG response includes two thumbnails, you've added 40-50 KB to every request. At 500,000 daily requests, that's an additional 25 GB/day of egress, or 750 GB/month ($67.50/month at AWS rates).

Embedding APIs at Scale

Float32 embeddings are surprisingly large. A 1536-dimension embedding vector (OpenAI text-embedding-ada-002 output dimension) encodes to 6,144 bytes in raw float32. Returning 1,000 embeddings per batch request sends 6 MB per response. At 100,000 embedding requests per day with average batch size of 100, that's 61 GB/day or 1.8 TB/month. Switching to float16 or int8 quantized embeddings halves or quarters this immediately.

Real-World Cost Model: 50M Tokens/Day

Here's a worked comparison for a streaming inference endpoint serving 50M output tokens per day on an 8x H100 SXM5 pod. This is a representative mid-scale production workload, roughly equivalent to a B2B API with 50-100K daily active users running sessions of average length.

Spheron H100 SXM5: $3.90/hr per GPU on-demand (from live API, 27 May 2026)

AWS P5.48xlarge (8x H100 SXM5): approximately $6.88/hr per GPU on-demand ($55.04/hr for the full 8-GPU instance). See AWS H100 pricing breakdown for current rates.

50M tokens/day egress calculation:

50,000,000 tokens/day × 4 bytes/token × 1.5x streaming overhead × 30 days
= 9,000,000,000 bytes/month
= ~8.4 GB/month outbound

For a text-only endpoint, egress itself is small. The compounding savings come from zero compute premium, no S3 egress for model weight loading, and no cross-AZ charges.

Note that the model weight loading line assumes the inference server pulls its weights from S3 on each container restart, a common pattern when using AWS ECS or EKS without persistent volumes. A 70B FP16 model is ~140 GB. If you restart once per day: 140 GB × 30 days × $0.09/GB = $378/month in S3 GET egress. Keeping model weights on a persistent volume attached to the inference node eliminates this entirely, regardless of cloud provider.

For video or multimodal workloads, swap the 8.4 GB egress figure for the actual payload sizes from the previous section. At 1.5 TB/month (video generation pipeline), the egress line becomes $135/month and grows from there.

Pricing fluctuates based on GPU availability. The prices above are based on 27 May 2026 and may have changed. Check current GPU pricing → for live rates.

If you've done a GPU cloud pricing comparison and found the compute delta already compelling, the egress analysis makes the case stronger for any workload that serves data at scale.

Cost-Cutting Techniques That Don't Require Migration

For teams committed to their current hyperscaler, these optimizations reduce egress without a provider change.

Response compression. JSON LLM responses compress aggressively: 60-80% reduction with gzip, slightly better with brotli. A 10 KB JSON response body compresses to 2-4 KB. If your inference API server (vLLM, TGI, or custom FastAPI) isn't already enabling gzip on responses, this is a one-line config change that immediately cuts egress volume. Most API gateways can handle this at the edge rather than at the inference server.

CDN fronting. Cloudflare's free tier caches static and semi-static responses with global edge PoPs. For deterministic inference endpoints (same prompt always produces the same output, which applies to zero-temperature completions and classification models), a 5-minute CDN TTL can serve the majority of repeat requests without hitting the origin. Even for non-deterministic endpoints, caching at edge for identical prompts within a short window (semantic cache) reduces origin traffic and eliminates the egress charge for cached hits.

Same-AZ deployment. AWS charges $0.01/GB in each direction for cross-AZ traffic. If your inference GPU is in us-east-1a and your application server is in us-east-1b, every response traverses two AZ boundaries. Moving to a same-AZ deployment eliminates this. For multi-AZ redundancy, the cost is unavoidable, but verify you actually need multi-AZ HA before paying for the traffic.

Batch asynchronous workloads. For non-interactive AI jobs (batch summarization, embedding generation, offline classification), streaming results back to the client in real time is unnecessary. Accumulate outputs and deliver them in a single response rather than SSE-streaming each token. This eliminates the 1.5-2x streaming multiplier and reduces connection count, which also cuts TCP overhead.

Avoid S3 model weight loading on every restart. Keep model weights on local NVMe or a persistent block volume attached to the inference instance. The first load is unavoidable, but every subsequent restart pulls from local disk instead of S3. For a 70B model on an instance that restarts daily, this saves ~$380/month in S3 egress alone.

Architecting to Minimize Egress

Regional Model Placement

Co-locate your inference GPU in the same region as your largest user concentration. AWS, GCP, and Azure all charge more for egress to certain geographies. Traffic from us-east-1 to Europe costs more than traffic staying within North America. If 60% of your users are in Europe, a Frankfurt or Paris inference endpoint may have lower effective egress costs than serving from Virginia, even accounting for any compute price differences between regions.

For the GPU cost optimization playbook, regional placement is step one before any further optimization. Routing users to the nearest inference node cuts both latency and egress charges.

Edge Inference

Quantized models at 4-bit or 8-bit precision run on consumer-grade GPUs at the network edge. A Llama 3 8B Q4 model fits in 5 GB of VRAM, runs adequately on an L40S or RTX 4090, and can be deployed in dozens of edge PoPs close to user concentrations. Egress from edge nodes typically qualifies for lower rates or is zero on platforms designed for edge workloads. The tradeoff is model quality and maintenance complexity.

For workloads where a smaller quantized model meets quality requirements, edge inference eliminates the egress charge and improves latency simultaneously.

Hybrid Topology

High-egress generation workloads (video, audio) run on a neocloud with zero egress fees. Orchestration, routing, and user-facing APIs stay on a hyperscaler or managed platform. The hybrid model concentrates bandwidth-intensive operations where egress is free and keeps stateful, compliance-sensitive workloads where the hyperscaler's governance features are valuable.

This architecture requires your inference endpoints to be addressable from outside the orchestration plane, which is standard for containerized deployments. The egress from orchestrator to inference node is internal and typically zero or near-zero on most platforms.

Migration Checklist: Moving Egress-Heavy Workloads

Moving an inference endpoint from a hyperscaler to a zero-egress neocloud is straightforward for stateless workloads. The process typically takes a few hours of engineering time for a well-containerized service.

1. Rank workloads by monthly egress GB. Use your cloud provider's billing export to identify the top 3-5 highest-egress services. Start with the highest-volume one where the compute configuration is simple (single GPU, stateless inference).

2. Containerize your inference stack. Package the model server (vLLM, TGI, or custom) with its runtime dependencies into a Docker image. Test locally with docker run before targeting a new provider. If you're already on Kubernetes or ECS, you likely have container images ready.

3. Externalize all state to S3-compatible object storage. The new provider needs to be able to access your model weights, checkpoints, and any persistent artifacts. Most neoclouds support S3-compatible endpoints or allow mounting volumes pre-loaded with weights. Confirm before migrating.

4. Benchmark on the target provider. Provision equivalent GPU capacity and run your standard load tests against the new endpoint. Compare tokens/sec, latency percentiles, and memory usage. Differences in NVLink topology, NIC bandwidth, or driver versions can affect inference throughput.

5. Shadow-deploy and validate output parity. Route a small percentage of production traffic to the new endpoint (or mirror production requests to it) and compare outputs. For non-deterministic endpoints, compare distributions rather than exact outputs. Run this for 24-48 hours before cutting over.

6. Cut over and decommission. Shift all traffic to the new endpoint, monitor for 24 hours, then terminate the original instance. For migrating from AWS to a neocloud, the full mechanics of switching load balancers and updating DNS are covered in detail.

For A100 instances on Spheron or an H100 GPU rental, you can provision capacity immediately without reservation queues or quota approval processes. For technical setup and configuration options, see the Spheron documentation.

Egress is a slow leak. On any single day the charge is small. Over six months of production traffic, it compounds into thousands of dollars that showed up nowhere in the GPU pricing comparison you made before launching. The fix is architectural: zero-egress providers eliminate the problem entirely, and the techniques above address it within a hyperscaler if migration isn't on the table.

Egress fees are the most invisible line item in a hyperscaler GPU bill. Spheron charges no egress fees on GPU rentals, which can save hundreds to thousands of dollars per month for inference endpoints serving millions of tokens daily. See the full GPU pricing → or start with an H100 →.

Get started on Spheron →