Self-Host a Text-to-SQL Data Analyst Agent on GPU Cloud (2026 Guide)

OpenAI's GPT-4o scores 86.6% execution accuracy on the original Spider text-to-SQL benchmark. Point the same model at Spider 2.0's real enterprise schemas, the ones with 1,000+ columns and multiple SQL dialects, and accuracy collapses to 10.1% (Spider 2.0). A bigger, smarter general-purpose model doesn't fix that. What fixes it is schema-aware retrieval plus a model actually trained to write SQL, which is exactly the architecture behind WrenAI, Vanna, and PremSQL.

This matters more than usual right now. The global AI copilot market, which includes BI and data-analyst copilots, is on pace to grow from $16.94B in 2025 to $21.59B in 2026, a 27.4% CAGR (The Business Research Company). Every one of those copilots needs to see your warehouse schema, and often your actual query results, to answer a question. Send that to a hosted API and you've handed a third party your table structure, your business logic embedded in column names, and whatever rows come back. Self-host the same pipeline, semantic layer, vector store, and a fine-tuned SQL model, on rented GPUs, and none of it leaves your VPC.

This guide covers why schema-aware retrieval beats a raw context dump, how WrenAI, Vanna, and PremSQL compare as your RAG and orchestration layer, which fine-tuned SQL model fits your accuracy bar, and exactly how much GPU you need at each model tier. For the shared retrieval infrastructure underneath any of these stacks, our agentic RAG on GPU cloud guide covers the embedding, vector search, and LLM colocation pattern this guide builds on directly.

Why Text-to-SQL Agents Need Schema Context, Not Just a Big Model

A text-to-SQL agent's hardest problem isn't language understanding, it's figuring out which 5 tables out of 400 actually answer the question. Get that step wrong and no amount of model quality saves the query.

The Spider 2.0 Reality Check: Enterprise Schemas Break Naive Prompting

Spider 2.0 tests 632 real-world text-to-SQL workflows pulled from actual enterprise use cases, with schemas that often exceed 1,000 columns and, in extreme cases, run past 3,000 (Spider 2.0). That's the gap between academic benchmarks and what an analyst actually deals with at a mid-size company running Snowflake or BigQuery with years of accumulated tables.

The accuracy numbers on that benchmark are the whole argument for this post. GPT-4o goes from 86.6% on Spider 1.0 to 10.1% on Spider 2.0. o1-preview, a reasoning model built specifically to think longer before answering, still only solves 17.1% of Spider 2.0 tasks (Spider 2.0). Reasoning capacity doesn't fix a context problem. If the model never sees the right 5 tables in the first place, no amount of chain-of-thought recovers the missing information.

Compare that to BIRD, an earlier and more contained benchmark: 12,751 question-SQL pairs across 95 databases, 33.4GB total, spanning 37+ professional domains (BIRD). The current leaderboard leader, AskData paired with GPT-4o, hits 81.95% execution accuracy against a 92.96% human baseline (BIRD). BIRD schemas are complex but bounded. Spider 2.0 schemas are not, and that's the difference that actually matters for production deployments.

Schema Linking vs. Dumping the Whole Schema in Context

The naive approach, paste every table and column definition into the system prompt, fails for three concrete reasons, not just one vague "context limit" excuse. First, a 1,000+ column schema simply doesn't fit most context windows once you add the question, few-shot examples, and conversation history. Second, even when it does fit, irrelevant tables are noise the model has to filter, and every filtering step is a chance to pick the wrong table. Third, and this is the one people underestimate, LLMs hallucinate relationships between tables and columns based on superficial name similarity rather than genuine semantic connections, a schema-linking failure that gets worse as schema size grows, which is why graph-based approaches to schema linking are an active research area (SchemaGraphSQL). A column named customer_id in your CRM schema and a column named customer_id in your billing schema are not automatically the same join key, but a model skimming a huge flat schema dump will happily assume they are.

Schema linking solves this by retrieving only the tables and columns relevant to the specific question before generation ever happens, the same recall-then-generate pattern that any RAG pipeline uses, just applied to schema metadata instead of documents. WrenAI's semantic engine and Vanna's vector-store retrieval both do this: narrow the schema first, generate second. Our self-hosted vector database guide covers deploying the Qdrant or Milvus backend this retrieval step depends on.

Open-Source Stack: WrenAI, Vanna, and a Fine-Tuned SQL-Coder Model

Three pieces make up a self-hosted text-to-SQL agent: a semantic or retrieval layer that narrows the schema, a model that turns the question plus that narrowed context into SQL, and, for PremSQL specifically, an execution and UI layer that closes the loop end to end.

WrenAI - Semantic Layer + RAG for Governed GenBI

WrenAI is the heaviest-duty option of the three, built around a proper semantic layer rather than raw retrieval alone. Its core, Wren Engine, is an Apache DataFusion-based engine written in Rust with Python bindings and a WebAssembly build, sitting on top of a Modeling Definition Language (MDL) that captures your actual business semantics, metric definitions, joins, naming conventions, instead of leaving that context to be re-derived by the model every query. On top of that sits a RAG-based query service and a UI, with LangChain/LangGraph integration for teams wiring WrenAI into a broader agent stack (WrenAI).

It supports 22+ data sources out of the box, including BigQuery, Snowflake, PostgreSQL, ClickHouse, Redshift, and Databricks, and is licensed Apache 2.0 for the core engine with AGPL-3.0 covering other components (WrenAI). At 16.6k GitHub stars and 1.9k forks, it's the most widely adopted option here by a clear margin (WrenAI). The tradeoff is setup weight: a semantic layer means modeling your business logic once upfront (worth it if multiple teams query the same warehouse and you want governed, consistent definitions across all of them), rather than skipping straight to question-in, SQL-out.

Vanna - Lightweight RAG Framework, Bring Your Own Vector Store and LLM

Vanna takes the opposite approach: no semantic layer, no bundled UI opinion, just a training-and-retrieval loop you wire into whatever LLM and vector store you already run. You train it by feeding DDL statements, business documentation, and past question-SQL pairs; each gets embedded and stored in a vector database of your choice (ChromaDB, Qdrant, and Milvus are all supported integrations), and at query time Vanna retrieves the most relevant training items and passes them to your LLM as context alongside the question (Vanna, Milvus integration docs).

It's MIT-licensed, which makes it the simplest to fold into an existing stack if you already run a vector database for other RAG use cases and just want the schema-linking layer, not a full platform. The cost of that flexibility is that you own more of the wiring yourself: model choice, vector store choice, and the training pipeline that keeps your DDL and documentation current as your schema evolves.

PremSQL - Local-First Pipelines and the Prem-1B-SQL Model

PremSQL is built for one specific case: teams that need every part of the pipeline, including the LLM, running locally with zero external API dependency. It's MIT-licensed and ships connectors for Ollama, HuggingFace, Apple MLX, PremAI, and OpenAI, plus an AgentServer FastAPI wrapper for deployment and a self-hosted Playground UI that gives you a ChatGPT-style interface for AI-powered data analysis without any of it touching a third-party server (PremSQL).

The model most closely tied to PremSQL is Prem-1B-SQL, a 1.3B-parameter model fully fine-tuned from DeepSeek-Coder-1.3B-Instruct. It reaches 85% execution accuracy on Spider and 51.54% on BIRD's private test set (46% on the public dev split), the first model under 2B parameters to break 50% on BIRD's private benchmark (Prem AI). For teams whose compliance requirement is "the model itself must never call out to anyone else's infrastructure," this is the tightest path available, and it's small enough to run alongside the embedding model and reranker on a single modest GPU, which we size out below.

Picking a Fine-Tuned SQL Model: SQLCoder-7B/70B vs. Arctic-Text2SQL-R1 vs. OmniSQL

Once you've picked a retrieval layer, you still need to choose the model that actually writes the SQL. None of these three are drop-in replacements for each other, they trade off differently on accuracy, VRAM, and licensing.

Defog's SQLCoder family is the most established. SQLCoder-7B-2 is fine-tuned from CodeLlama-7B and needs about 12.6GB of VRAM at fp16/bf16, 6.3GB at int8, or 3.1GB at int4, plus roughly 20% overhead for inference (HuggingFace discussion). SQLCoder-70B-Alpha, fine-tuned from CodeLlama-70B, is a substantially different tier: on Defog's own SQL-Eval benchmark it hits 97.1% on joins and order-by clauses versus 74.3% for the original 7B model, and 91.4% versus 74.3% on WHERE clauses, with the model card stating it "outperforms all generalist models, including GPT-4, on text to SQL" (HuggingFace model card). That accuracy comes at a real VRAM cost: roughly 138GB at full FP16 (69B params at 2 bytes each), or about 41.4GB as a Q4_K_M GGUF quant (tensorblock GGUF quants).

Snowflake's Arctic-Text2SQL-R1 takes a different training approach entirely: GRPO reinforcement learning with a simple execution-based reward, rather than supervised fine-tuning on labeled examples. It ships in 7B, 14B, and 32B sizes, with the 7B variant confirmed at 68.9% execution accuracy on BIRD-dev and the 32B variant reported around 71.8%, putting it ahead of several larger open and proprietary models at release (Snowflake Engineering Blog, HuggingFace model card). The appeal here is efficiency: a 7B model trained with execution-based rewards closing most of the gap to models several times its size.

OmniSQL-32B, fine-tuned from the Qwen2.5-Coder family on the SynSQL-2.5M dataset (2.5 million examples across more than 16,000 databases) plus the Spider and BIRD training sets, is a current open-source option in the 32B class that the model card positions as competitive with GPT-4o and DeepSeek-V3 on BIRD (OmniSQL model card). If you're already standardized on Qwen-family models elsewhere in your stack, per our VRAM tier guide for self-hosted LLMs, OmniSQL-32B is the natural SQL-specific complement rather than introducing a second model family.

For most teams, the practical default is SQLCoder-7B-2 or Arctic-Text2SQL-R1-7B paired with real schema retrieval, moving up to SQLCoder-70B-Alpha or a 32B model only once you've confirmed retrieval quality is not the bottleneck, since a bigger model can't compensate for the wrong tables landing in context in the first place.

GPU Sizing for Low-Latency Query Generation at Analyst Scale

Size the GPU around the model tier you pick, plus the embedding model and reranker that sit in front of it, not around raw ticket or query volume. Analyst-facing text-to-SQL is a low-concurrency, latency-sensitive workload: a handful of analysts firing off queries and waiting seconds for SQL back, not thousands of concurrent chat sessions.

VRAM Budget by Model Tier (7B, 32B, 70B)

The 32B and 70B AWQ/GGUF figures track the same quantization behavior our fine-tuning VRAM guide documents for other 32B-class open models: INT4 quantization consistently lands in the 16-18GB range for 32B models, roughly a quarter of full FP16 weight size. Note the 70B tier's GGUF path typically means serving through llama.cpp rather than vLLM, since vLLM's AWQ/FP8 quantization paths are less mature for 70B+ CodeLlama-family checkpoints than for the more common Llama and Qwen families.

Where the Embedding Model and Reranker Fit Alongside the SQL Model

Schema linking needs an embedding model to vectorize table/column descriptions and past query pairs, and ideally a reranker to re-score retrieved candidates before they hit the SQL model's context. Both are small relative to the SQL model itself: a BGE-M3 or Qwen3-Embedding model runs 1-3GB, and a cross-encoder reranker adds another 1-2GB. Our self-hosted embeddings and rerankers guide covers deploying these with Hugging Face's TEI server, including throughput tuning if your warehouse has enough tables that retrieval latency starts to matter.

Budgeting the full stack: a 7B SQL model plus embedding plus reranker lands around 15-18GB at fp16, comfortably inside a single 48GB card with room left for KV cache. A 32B SQL model at AWQ INT4 plus the same retrieval components lands around 20-22GB, again leaving headroom on an 80GB card. The pattern holds across tiers: the SQL model dominates the VRAM budget, retrieval components are a rounding error on top.

Recommended GPUs on Spheron by Stack Size

Pricing fluctuates based on GPU availability. The prices above are based on 24 Jul 2026 and may have changed. Check current GPU pricing → for live rates.

The L40S tier is the right starting point for most teams: it's enough VRAM for a strong 7B model with real schema retrieval, and the accuracy ceiling of a well-grounded 7B model on your actual schema will usually beat a 70B model with no retrieval at all. Move up a tier only once retrieval is solid and the remaining errors are genuinely model-capability errors, not schema-selection errors.

We tested this on a single L40S: SQLCoder-7B-2 served through vllm serve defog/sqlcoder-7b-2 --dtype bfloat16 --max-model-len 4096, with Vanna handling retrieval against a 14-table e-commerce schema (orders, order_items, customers, products, inventory, and a handful of lookup tables). Asked "what's the average order value by region for the last two full quarters," Vanna's retrieval step narrowed those 14 tables down to four, orders, order_items, customers, and a region lookup, before the model ever saw a prompt. SQL came back in under 3 seconds end to end on a cold server with no request batching, which lines up with what you'd expect from a 7B model at this VRAM budget. Skipping retrieval and pasting the full 14-table DDL into the same prompt instead produced a query joining customers to products on a shared id column, syntactically valid, semantically wrong. That's the same name-similarity hallucination from earlier, reproduced in a five-minute test rather than an academic benchmark.

Keeping Warehouse Credentials and Query Results Off Third-Party APIs

77% of organizations identify intellectual property protection of AI datasets as a top concern, and 90% have expanded their privacy programs specifically because of AI, according to Cisco's 2026 Data and Privacy Benchmark Study (Kiteworks). A text-to-SQL agent is a direct pipeline for exactly that kind of exposure, because the whole point of the tool is feeding it your schema and your data.

What Actually Leaves Your Network When You Call a Hosted Text-to-SQL API?

Walk through a single query against a hosted text-to-SQL API and here's what crosses the wire: the schema metadata needed for retrieval (table names, column names, and often sample rows so the model can infer data types and formats correctly), the analyst's natural language question (which frequently encodes business context, "show me the churn numbers before the pricing change" tells a competitor plenty on its own), the generated SQL itself, and, if the tool executes the query and summarizes results rather than just handing back raw SQL, the actual query output. That last piece is easy to overlook. A tool that just generates SQL text is lower risk than one that also runs it and reports "Q3 revenue was $4.2M, down 8% from Q2," because the second version has now transmitted an actual business result to a third party's inference endpoint.

Self-Hosted Architecture: Semantic Layer, Vector Store, and LLM in One VPC

The self-hosted version of this stack keeps all four of those pieces, semantic layer, vector store, SQL model, and query execution, inside infrastructure you control. WrenAI's Wren Engine or Vanna's training pipeline sits in front of a vector store that retrieves the relevant schema slice for the SQL model to generate against, the same colocation pattern covered in our vector store and agentic RAG guides linked above: embedding model, vector index, and LLM inference on the same GPU node, cutting network round trips to near zero while keeping every component under your own control. The warehouse connection itself, credentials included, never needs to leave your network either, since the agent connects directly to Snowflake, BigQuery, or Postgres from inside your VPC rather than routing through a hosted intermediary that needs its own copy of your connection string. See Spheron's docs for provisioning details if you're scripting this deployment as part of a larger pipeline.

FAQs

What's the difference between WrenAI, Vanna, and PremSQL for a self-hosted text-to-SQL agent?

WrenAI gives you a full semantic layer plus RAG query service across 22+ data sources, the right choice if multiple teams need governed, consistent business definitions. Vanna is a lighter MIT-licensed framework where you bring your own vector store and LLM, best if you already run RAG infrastructure and want the schema-linking layer added on top. PremSQL is the fully local-first option, an AgentServer and Playground UI plus SQL model that can run without a single external API call, the tightest fit for strict data-residency requirements.

Do I need a fine-tuned SQL model, or can I just use a general-purpose LLM?

At enterprise schema scale, general models fall apart badly: GPT-4o drops from 86.6% on Spider 1.0 to 10.1% on Spider 2.0's real enterprise workflows, and o1-preview only reaches 17.1%. A fine-tuned SQL-Coder-class model paired with schema retrieval closes most of that gap.

What GPU do I need to self-host a text-to-SQL data analyst agent?

A 7B model like SQLCoder-7B-2 fits a single L40S 48GB with room for an embedding model and reranker. A 32B model at AWQ INT4 fits an A100 80GB or H100 PCIe. SQLCoder-70B-Alpha needs either a ~41GB Q4_K_M GGUF quant on one H100 80GB, or roughly 138GB at full FP16 across two H100s.

Does self-hosting actually keep warehouse credentials and query results private?

Yes. Every hosted text-to-SQL API call carries your schema, your question, the generated SQL, and often the actual query results. Self-hosting keeps all of that inside your own VPC, directly relevant given 77% of organizations now identify IP protection of AI datasets as a top concern.

Can this stack run fully offline with no external API calls at all?

Yes, with PremSQL. Its Ollama, HuggingFace, and Apple MLX connectors, plus a self-hosted AgentServer and Playground, mean the LLM itself can run locally with zero external dependency.

A text-to-SQL agent only earns its keep once analysts trust it with real schemas and real data, which means the schema, the questions, and the results all need to stay inside your own infrastructure. Spheron's per-minute billing means the GPU cost only accrues while analysts are actually querying, not around the clock.

Spheron's A100 instances → | Check L40S availability → | View all GPU pricing →