NVIDIA L4 vs L40S: Cheapest GPU for AI Inference (2026)

The NVIDIA L4 and L40S share an architecture, a naming scheme, and almost nothing else that matters for a production inference budget. The L4 has 24GB of GDDR6 and draws 72W. The L40S has 48GB of GDDR6 ECC and draws 350W, nearly five times as much power for twice the memory. On Spheron, the L40S runs $0.96/hr on-demand right now; RunPod lists an L4 at $0.39/hr, less than half that. Teams that don't check the spec sheet tend to default to the L40S "to be safe," and for a meaningful slice of workloads, that caution is just wasted spend.

We break down where that default is justified and where it isn't: the full spec comparison, live pricing across four providers, cost-per-token math at realistic batch sizes, and the VRAM and concurrency thresholds that actually separate an L4 workload from an L40S one. If you land on needing the bigger card, the L40S inference guide has the deeper benchmark breakdown and deployment walkthrough.

NVIDIA L4 vs L40S: Specs, Power Draw, and VRAM Compared

Both cards are Ada Lovelace, but they come off different dies. The L4 uses the AD104 die, the same mid-range silicon family behind several RTX 40-series consumer cards. The L40S uses the AD102 die, NVIDIA's largest Ada Lovelace chip and the same one under the L40 and RTX 6000 Ada Generation.

That die difference is the whole story. The L4 is a compact, low-power inference and video card built to slot into any 1U server. The L40S is a full-size data center accelerator that happens to share a naming convention and an architecture generation with the L4, nothing more.

Full Spec Table

Source: NVIDIA's official L4 data sheet (nvidia.com/en-us/data-center/l4) and this site's L40 vs L40S spec comparison, which pulls from NVIDIA's L40S data sheet.

On paper, the L40S delivers roughly 3x the raw FP8 throughput of the L4 (733 vs 242 TFLOPS dense), 2x the VRAM, and nearly 3x the memory bandwidth, according to a side-by-side breakdown from AceCloud's L4 vs L40S comparison (acecloud.ai). None of that is in dispute. The question that actually matters is whether your workload uses any of that extra headroom, because you pay for all of it whether you use it or not.

Why the Names Get Confused

Both cards launched in the same generation, both carry the letter "L," and both target inference over training. NVIDIA's naming scheme doesn't signal the tier gap the way "RTX 4060" versus "RTX 4090" does for consumer cards. A buyer skimming a provider's GPU list sees "L4" and "L40S" sitting next to each other alphabetically and reasonably assumes they're a small step apart, not a different die, a 4x power budget, and roughly a 3x price gap.

NVIDIA announced the L4 on March 21, 2023 at GTC as a universal entry-level data center GPU aimed at video, graphics, and light AI inference, according to NVIDIA's official launch announcement. The L40S launched later that year as the inference-tuned sibling of the L40 workstation card, positioned to fill the gap between the L4 and the H100 for teams that needed real LLM throughput without H100 pricing. The two were never meant to be cross-shopped as equivalent options. They were built for adjacent but distinct tiers.

Power Draw Math: How Many L4s Fit in an L40S's Power Budget

At 72W TDP, the L4 draws less power than a typical household light bulb and is the most power-efficient data center GPU NVIDIA makes, according to Jarvislabs' breakdown of L4 pricing and efficiency (jarvislabs.ai). Run the math against the L40S's 350W envelope and roughly 4.8 L4 cards fit in the power budget of a single L40S. The same source notes that about 5 L4 GPUs fit in the power draw of one 400W A100, and nearly 10 fit inside one 700W H100's envelope.

That density is the real argument for L4 in edge, colocation, or any deployment where rack power is the binding constraint rather than raw throughput. A rack that can run 8 L40S cards at 350W each (2,800W of GPU alone) could instead run around 38 L4 cards for aggregate throughput on small models that beats what 8 L40S cards deliver on the same models, at a fraction of the cooling load.

Cost Per GPU-Hour: L4 vs L40S Across Cloud Providers

The L4 costs meaningfully less than the L40S on every provider that lists both, but the gap varies a lot by vendor, and not every provider offers an L4 at all. Spheron currently rents the L40S but has no native L4 SKU on the platform.

Multi-Provider Pricing Table

Sources: instances.vantage.sh AWS g6.xlarge and g6e.xlarge, cloudprice.net GCP g2-standard-4, RunPod pricing, Jarvislabs L4 pricing guide.

Pricing fluctuates based on GPU availability. The prices above are based on 13 Jul 2026 and may have changed. Check current GPU pricing → for live rates.

Notably absent from this table: Azure. Azure's GPU-backed N-series VMs run A10, A100, and H100 (with H200 and GB200 on the newest ND-series SKUs), with no native L4 or L40S family as of this writing, according to Thunder Compute's Azure GPU instance breakdown (thundercompute.com). If your infrastructure is Azure-committed, neither of these cards is on the menu without going through a third-party marketplace listing.

On-Demand vs Spot Pricing for Both GPUs

GCP's g2-standard-4 on-demand rate of $0.7068/hr drops to roughly $0.344/hr on Spot/preemptible pricing, a 51% discount according to pricing data compiled by Economize Cloud. That's the clearest documented spot discount for L4 in this comparison. AWS and RunPod don't split L4 pricing into a distinct on-demand-versus-spot line the way GCP does; RunPod's Community Cloud rate already sits closer to what other providers treat as a discounted tier.

Spot and preemptible capacity comes with the standard caveat: it can be reclaimed with little to no notice, so it's a fit for batch embedding jobs and offline evaluation runs, not for a customer-facing endpoint with a latency SLA. For inference that needs to stay up, budget on-demand pricing and treat spot as a cost lever for anything that can tolerate interruption.

Cost-Per-Million-Tokens Math at Realistic Batch Sizes for 7B-8B Models

Here's where the L4's lower hourly rate either pays off or doesn't, depending on how you're actually serving the model. We're using measured L40S throughput for Llama 3.1 8B from this site's L40S inference benchmarks and Spheron's current $0.96/hr on-demand rate:

Rates as of 13 Jul 2026; check current GPU pricing → before budgeting off these numbers.

The L4 doesn't have an equivalent published benchmark on this site yet, so here's the back-of-envelope version, scaled from the L40S numbers above using each GPU's verified specs rather than guessed throughput. At batch size 1, decode-phase inference is memory-bandwidth-bound, and the L4's 300 GB/s is 34.7% of the L40S's 864 GB/s. Scaling the L40S FP16 batch-1 figure by that ratio puts L4 at roughly 16 tok/s. At batch 8, the workload shifts toward compute-bound, and the L4's FP8 dense throughput (242 TFLOPS) is about 33% of the L40S's (733 TFLOPS), putting L4 FP8 batch-8 throughput at roughly 166 tok/s.

Plugging those estimates into RunPod's $0.39/hr L4 rate:

These are estimates, not measured throughput, and the underlying rates move with availability. RunPod's $0.39/hr figure is current as of 13 Jul 2026; check current GPU pricing → for live rates before you commit to a number.

The pattern holds across both cards: cost per token drops sharply as you fill the batch, and the L4's lower hourly rate mostly offsets its lower throughput rather than beating it outright. At batch 1, L4 is actually a bit more expensive per token in this estimate because its throughput drop outpaces its price drop. At batch 8, the two cards land close together. The L4's real advantage isn't raw cost-per-token at any single batch size, it's running many small, independent instances in parallel without a 350W-per-card power bill. For a deeper cross-model breakdown of this math, see the GPU cost-per-token benchmark guide.

Which Inference Workloads Actually Need L40S Over L4

The specs and pricing tables only matter in the context of what you're actually deploying. Here's how we'd split it by workload.

Where L4 Wins

• Embeddings and classification. These models are typically under 1B parameters, run at high request rates, and barely touch the L4's 24GB. You're paying for throughput per dollar, not headroom, and the L4 wins on both hourly cost and power efficiency.
• 7B-8B chat models at INT4 or FP8. A quantized Llama 3.1 8B or Qwen 2.5 7B fits comfortably in 24GB with room for KV cache at moderate concurrency. This is the L4's home turf.
• Dense, edge, or colocation deployments. When you're power- or rack-constrained rather than throughput-constrained, the L4's 72W TDP lets you pack far more inference capacity into the same power and cooling budget, as the density math above shows.
• Video inference pipelines that don't chain multiple large models. The L4's NVENC/NVDEC hardware handles standard encode/decode workloads at a fraction of the L40S's power draw.

Where L40S Is Worth the Premium

• 13B-34B models without heavy quantization. These need more VRAM headroom than 24GB comfortably gives you once KV cache is factored in at real context lengths. The VRAM tier guide for self-hosting LLMs maps model size to VRAM tier directly.
• Higher concurrency serving. Once you're running batch sizes above 8-16 concurrent requests against a single model, the L40S's 864 GB/s bandwidth stops being optional. This is the same threshold covered in the L40S vs H100 cost-per-token guide for the next tier up, and we've run the same cost-per-token comparison against Ampere in our L40S vs A100 guide if you're weighing older-generation capacity instead.
• Video and multimodal pipelines running several models at once. A pipeline chaining a vision encoder, an LLM, and a diffusion decoder on one card needs the extra 24GB just to hold everything resident.
• Fine-tuning or light training on top of inference. The L40S's higher compute ceiling and larger VRAM budget make it workable for LoRA fine-tuning jobs that would be tight on an L4. The GPU VRAM requirements guide for fine-tuning has the full sizing tables.

If VRAM is the constraint but the model is otherwise a good fit for a smaller card, quantization is usually the cheaper fix before you upgrade hardware. The AWQ quantization guide walks through cutting a model's memory footprint by roughly half with minimal quality loss, which is frequently what makes a 13B model workable on 24GB instead of forcing a jump to L40S.

Decision Matrix by Workload

For a broader view of where both GPUs sit against H100, H200, and B200, the best GPU for AI inference guide covers the full stack, and the best NVIDIA GPUs for LLMs guide has additional L4 pricing context consistent with the figures here.

If your workload lands in the L4 column above, quantize first and check whether your existing GPU budget already covers it before paying for L40S headroom you won't use. If it lands in the L40S column, per-minute billing means you can test the real throughput on your actual model before committing to a monthly spend.

Rent L40S on Spheron →