Google Cloud doesn't mark up Claude. It marks up the open-weight models sitting next to Claude in the same catalog, and that split is the whole story behind Vertex AI Model Garden pricing in 2026: two very different rate cards running through one product.
Model Garden hosts more than 200 foundation models spanning Google's own Gemini and Gemma families alongside third-party models including Anthropic Claude, Meta Llama, and Mistral, all behind one API surface and one billing account. That "one platform" pitch is deliberate. At Google Cloud Next 2026, CEO Thomas Kurian drew a direct line between Google's full-stack approach and its rivals: "Other vendors are handing you the pieces, not the platform." Model Garden is the piece of that platform that actually shows up on your invoice, and it prices very differently depending on which model you point at.
Claude on Vertex costs what Claude costs everywhere else, until you need a region-pinned endpoint. Llama costs meaningfully more than the same weights cost on a dedicated inference provider, and that markup is baked into the token rate with no separate line item to point at. This post walks through what Vertex actually charges by model, where the Llama markup comes from, the idle-endpoint cost Google doesn't put on the pricing page, and the real math on when renting GPUs and running the weights yourself beats Model Garden's price.
Vertex AI Model Garden Pricing: What You Actually Pay Per Model
Model Garden isn't one price list. It's three different billing models stapled together depending on which model you pick: Claude bills at Anthropic's own rate, most open-weight models bill per token at a Google-set rate, and some models can be self-deployed to a dedicated endpoint billed by the GPU-hour instead. Knowing which bucket your model falls into matters more than any single number on the pricing page.
Claude on Vertex AI: Same Token Price as Anthropic's Direct API, With a Regional Catch
Anthropic's own pricing documentation is explicit that Google Cloud is a "partner-operated" platform with independent regional pricing, and it points to Google's own pricing page as the source of truth rather than listing Vertex rates itself. In practice, that independent pricing tracks Anthropic's direct API dollar for dollar on the base token rate. Claude Sonnet 5 lists at $2 per million input tokens and $10 per million output tokens on Anthropic's own API, and Vertex AI is priced to match on the global endpoint.
The catch is routing, not the token rate. Starting with Claude Sonnet 4.5, Haiku 4.5, and Opus 4.5, Google Cloud offers three Claude endpoint types: global (dynamic routing for maximum availability), multi-region (routing within a geographic area), and regional (pinned to one region). Regional and multi-region endpoints carry a 10% premium over the global default. If your compliance team needs Claude inference to stay inside the EU or a named region, that guarantee costs 10% more than letting Vertex route wherever it wants, the same shape of tradeoff AWS Bedrock applies to its own regional Claude endpoints. Since Claude isn't open-weight, this API-vs-API comparison is really the only lever you have; our Claude Opus 4.8 API vs self-hosted LLMs breakdown covers the separate question of when an open-weight model on rented GPUs beats Claude entirely, rather than just picking which cloud serves Claude.
Llama and Third-Party Open Models: The Markup Is Baked Into the Token Rate
Llama, Mistral, and the rest of Model Garden's third-party open-weight catalog don't get Claude's pricing parity. Google sets its own per-token rate for these models, and that rate runs well above what the identical weights cost on a dedicated inference provider. There's no separate "Google tax" line item on the invoice. The markup is just the token price itself, which makes it easy to miss unless you check a competing provider's rate for the same model side by side.
Self-Deploy Open Models: GPU Node-Hours Plus a Management Fee
A subset of Model Garden's open models can also be deployed to a dedicated Vertex endpoint instead of called through the shared per-token API. That path bills differently: you pay for the underlying GCP compute (A100, H100, L4, T4) by the hour, plus a separate Vertex AI infrastructure and management fee stacked on top of the raw GPU rate. In us-central1, A100 40GB runs about $2.93/hr in compute plus roughly $0.44/hr in management fees, for about $3.37/hr all-in. H100 80GB runs about $9.80/hr in compute plus roughly $1.47/hr in management fees, landing around $11.27/hr. L4 runs about $0.64/hr. Those figures move with GCP's own list pricing, so treat them as directional rather than a quote you can budget against without checking the live console.
Worth noticing: Vertex's H100 self-deploy compute rate ($9.80/hr) sits close to what our own Google Cloud A3 H100 pricing breakdown measured for GCP's raw A3 on-demand instances (~$10.98/hr per GPU as of that post's date). The management fee is what's new here, a charge for the Model Garden deployment layer on top of compute you could otherwise rent directly.
The Markup: Vertex AI Model Garden Pricing vs Dedicated Inference
Claude's parity story doesn't extend to the rest of the catalog. For the open-weight models Model Garden hosts, the same weights are available from dedicated inference providers at a materially lower price, and the gap is large enough to matter at any real volume.
Llama 70B: Vertex's ~55% Premium Over Together AI
Llama 3.3 70B costs $1.36 per million tokens on Vertex AI Model Garden. The identical open-weight model costs $0.88 per million tokens on Together AI, a roughly 55% premium for running the exact same weights inside Google's managed layer. Nothing about the model changes between the two platforms. What you're paying the extra 55% for is Vertex's infrastructure, billing integration, and the convenience of one API surface across 200+ models, not anything specific to how well Llama runs.
That premium is real, but it's worth putting in context: it's the smaller of the two hyperscaler markups on this exact model. For the same Llama models, AWS Bedrock's premium over Together AI runs around 201%, more than three times Vertex's markup. If your comparison set is "Vertex vs Bedrock" rather than "managed API vs dedicated inference," Vertex is the cheaper hyperscaler for this model, just not the cheapest option available.
Llama 4 Scout and the Smaller-Model Markup Gap
Llama 4 Scout costs $0.25 per million input tokens and $0.72 per million output tokens on Vertex AI, against $0.18 and $0.59 on Together AI, a roughly 39% premium. The percentage markup on Scout is smaller than on 70B, but that percentage almost doesn't matter here. Scout is a 17B-parameter mixture-of-experts model, which means it fits comfortably on a single GPU rather than the 8-GPU footprint a 70B dense model needs. Our own cost-per-token benchmark puts Llama 4 Scout on a single H100 SXM5 at 4,200 tokens/second via vLLM, working out to about $0.19 per million tokens on-demand, well under either hyperscaler's price regardless of the exact markup percentage. For small and MoE models, the question isn't "is the API markup 39% or 55%," it's whether the model is cheap enough to self-host at all, and Scout clears that bar easily.
Why the Same Open Weights Cost More Inside Model Garden
The mechanics are straightforward once you separate what you're actually paying for. A dedicated inference provider like Together AI runs one thing: serving open-weight models at the highest throughput it can extract per dollar of GPU time, and it prices close to that cost. Vertex AI is pricing a bundle: the token rate for Llama also funds the shared API surface across 200+ models, integration with the rest of Google Cloud's IAM, logging, and billing stack, and the SLA and support tier that comes with a hyperscaler contract. None of that changes what the model computes. It changes what you're buying alongside the computation, and for a team that just wants Llama 3.3 70B tokens as cheaply as possible, that bundle is the 55% you're not getting back.
The Cost Vertex Doesn't Put on the Pricing Page: Idle Endpoints
Every dollar figure above assumes you're only paying for tokens you actually use. Self-deployed Model Garden endpoints don't work that way. Vertex AI does not support automatic scale-to-zero for deployed models: once you deploy a model to an endpoint, charges accrue continuously until you explicitly undeploy it, independent of how much traffic the endpoint serves.
That gap between "deployed" and "in use" has a documented real-world cost. One team deployed three e2-standard-4 endpoints at $0.154/hr each for A/B testing, paused the test, and forgot to undeploy them. Thirty days later, the bill showed $332.64 in fees for endpoints that served zero production traffic ($0.154/hr x 3 endpoints x 720 hours). That example uses small CPU-backed endpoints; run the same forgotten-endpoint scenario on a self-deployed H100 endpoint at $11.27/hr all-in, and 30 idle days costs over $8,100, not $332. Provisioned Throughput on AWS Bedrock has the same "you pay for the reservation, not the usage" shape, but at least it's a deliberate reservation you chose to make. A Vertex endpoint you forgot to tear down is the same bill with none of the intent.
The practical fix is process, not pricing: treat every Model Garden self-deploy endpoint like a running GPU instance you're renting by the hour, because that's exactly what it is. Set a calendar reminder or a budget alert on the project, not just on the model.
When Self-Hosting Breaks Even
The Llama markup makes self-hosting look like the obvious answer. The real math is closer than that, and it depends heavily on which precision you're willing to run.
Monthly Token Volume Where a Rented GPU Cluster Beats Model Garden
Take Llama 3.3 70B as the reference case, since it's the model with both a clear Vertex price ($1.36/M tokens) and published self-hosted throughput numbers. An 8x H100 SXM5 cluster needed to run 70B in FP16 currently costs about $27.04/hr on Spheron on-demand, or roughly $19,469/month at 720 hours. Our internal cost-per-token benchmark measured that cluster serving Llama 3.3 70B via vLLM with continuous batching at three precisions:
| Precision | Throughput (tok/s) | Self-hosted CPM | vs Vertex's $1.36/M |
|---|---|---|---|
| FP16 | 2,800 | ~$2.68 | 97% more expensive |
| FP8 | 5,600 | ~$1.34 | About 1.5% cheaper |
| INT4 (AWQ) | 8,400 | ~$0.89 | About 34% cheaper |
At FP16, self-hosting this model on a rented H100 cluster is never cheaper than Vertex's token rate, full stop, no volume threshold changes that. The cluster's cost per token is simply higher than what Google charges. FP8 brings self-hosting to roughly break-even, and here the volume question actually matters: dividing the cluster's $19,469 monthly fixed cost by Vertex's $1.36/M rate puts the breakeven at about 14.3 billion tokens a month, or 477 million tokens a day. An FP8 cluster's maximum throughput, running flat out at the 5,600 tok/s the benchmark measured, tops out at about 484 million tokens a day. That means hitting break-even on FP8 requires sustaining the cluster at roughly 98% of its absolute throughput ceiling, 24 hours a day, which is a much narrower margin than "self-hosting wins" implies. INT4 gives you real room: its 8,400 tok/s ceiling works out to about 726 million tokens a day, so the same breakeven volume needs only about 66% utilization, a target a genuinely high-traffic production workload can realistically sustain.
The honest takeaway: for a dense 70B model, self-hosting only clears Vertex's markup with quantization, and FP8 clears it by a margin thin enough that it's really a wash once you account for engineering time. INT4 is where self-hosting a large dense model like this becomes unambiguously cheaper, provided the accuracy tradeoff (2-4% on task-dependent evals, per the same benchmark) is acceptable for your use case.
Pricing fluctuates based on GPU availability. The prices above are based on 26 Aug 2026 and may have changed. Check current GPU pricing → for live rates.
Vertex AI Model Garden vs Self-Hosted: Decision Framework
| Use Vertex AI Model Garden when | Self-host on rented GPUs when |
|---|---|
| You're serving Claude, where Vertex matches Anthropic's direct API rate | You're serving a large dense open-weight model and can run FP8 or INT4 without an unacceptable quality hit |
| Volume is moderate and doesn't justify managing your own inference stack | Sustained traffic can keep the cluster near its throughput ceiling, not just near its VRAM ceiling |
| You need the 200+ model catalog and one billing surface across Gemini, Gemma, Claude, and Llama | The model is small or MoE (Scout-sized), where self-hosting undercuts either hyperscaler by a wide margin |
| Your workload is bursty and a self-deployed endpoint's always-on billing would sit idle most of the day | You'd rather own a transparent hourly GPU rate than a bundled per-token price funding Google's shared API layer |
| You want native IAM, logging, and Vertex Pipelines integration without standing up your own MLOps | Fine-tuning or continued pretraining is part of the roadmap; see the LLM fine-tuning cost guide for the rented-GPU side of that comparison |
Picking a model to self-host isn't just about clearing the Vertex price. Our VRAM tier guide for self-hosting open-source LLMs walks through which model actually fits your GPU budget before you commit to a quantization strategy. And if the migration is really "we're leaving a managed API behind," the playbook looks the same regardless of which API you're leaving, as covered in our OpenAI-to-self-hosted migration guide.
If Vertex's Llama pricing or an idle self-deploy endpoint is pushing your bill higher than expected, price the same model on hardware you control instead.
Frequently Asked Questions
No, not at the token-rate level. Claude models on Vertex AI bill at Anthropic's standard per-model rate, the same $2/$10 per million input/output tokens Claude Sonnet 5 lists at on Anthropic's own API. The cost that does climb is routing: starting with Claude Sonnet 4.5, Haiku 4.5, and Opus 4.5, Google Cloud's regional and multi-region Claude endpoints carry a 10% premium over the global default, per Anthropic's own pricing documentation.
Llama 3.3 70B runs $1.36 per million tokens on Vertex AI Model Garden, against $0.88 per million tokens for the identical open weights on Together AI, a roughly 55% premium for hosting the same model inside Google's managed layer.
No. Vertex AI does not support automatic scale-to-zero for models deployed to an endpoint. Charges accrue continuously from the moment you deploy until you explicitly undeploy it, regardless of how much traffic the endpoint actually serves.
For Llama 3.3 70B specifically, only once you apply FP8 or INT4 quantization on rented GPUs. At FP16 precision, a self-hosted H100 cluster's cost per million tokens runs higher than Vertex's on-demand rate. FP8 brings self-hosting to roughly parity, and INT4 pulls meaningfully ahead, but only if you can sustain enough traffic to keep the cluster near its throughput ceiling. Below that volume, Vertex's per-token pricing is the cheaper option.
