OpenAI's o3-deep-research bills $10 per million input tokens and $40 per million output tokens, on top of search fees it won't let you disable. Perplexity's unlimited Deep Research tier is $200 a month. Meanwhile, LangChain's Open Deep Research and Assaf Elovic's GPT Researcher already do most of what those products do, they're open source, and nobody has written the guide for running either one on your own GPUs.
This post covers what a deep research agent actually needs to run well: which open-weight model to use as the backbone, how much VRAM each role in the pipeline costs, why parallel sub-agent fan-out multiplies your token bill in a way a normal chat agent never does, and the actual deployment steps on rented GPU hardware. It closes with the real cost math against hosted Deep Research pricing, worked out with live numbers rather than guesses.
If you've already looked at self-hosting Perplexity-style AI search, this is a related but distinct workload. AI search answers one query with one retrieval pass. A deep research agent runs a multi-step investigation with several rounds of search, delegation, and synthesis, and that difference changes almost everything about how you size the GPU.
What a Deep Research Agent Actually Does Differently From AI Search
A search engine, even an AI-powered one, does one thing: it takes a query, retrieves some documents, and generates one grounded answer. A deep research agent takes a query, decides how to break the question apart, runs multiple independent research threads in parallel or in sequence, and only then writes a report. The output isn't a paragraph with citations, it's a structured document that reads like something a research analyst would produce over an afternoon.
LangChain put it plainly in the post announcing Open Deep Research: "Research is an open-ended task; the best strategy to answer a user request can't be easily known in advance." That single sentence explains the entire architecture. You can't hard-code a retrieval pipeline for an open-ended question the way you can for a search box, so both major open-source frameworks build a planning layer whose only job is figuring out what to research and how to split that work up.
Planner/Executor and Supervisor/Sub-Agent Patterns
Open Deep Research and GPT Researcher converge on the same basic shape with different names. Open Deep Research uses a supervisor that, per LangChain's own description, "has a simple job: delegate research tasks to an appropriate number of sub-agents." Each sub-agent gets an isolated context window and works its assigned sub-topic independently, which is what lets the system parallelize instead of researching everything in one long serial chain. GPT Researcher's planner-executor-publisher pipeline does the same conceptual job with a flatter hierarchy: a planner agent generates the research questions up front, execution agents fan out to gather sources for each question (typically pulling from 10 to 30 sources per report, per its own documentation), and a publisher agent aggregates everything into the final cited document.
The practical difference is how much the delegation decision itself costs. Open Deep Research's supervisor reasons about completeness and can spawn additional sub-agents mid-run if it decides the initial breakdown wasn't sufficient. GPT Researcher's planner front-loads the decomposition once, which is cheaper per report but less adaptive to how the research actually unfolds. Both patterns show up across the broader agent ecosystem covered in our multi-agent GPU infrastructure guide, which digs into orchestration topologies and KV cache math at scale beyond just research agents.
Why Parallel Sub-Agents Burn Far More Tokens Than a Chat or a Single Search Query
This is the number that should actually drive your GPU sizing decision. Anthropic built the multi-agent research architecture that both open-source projects mirror, and their own engineering write-up is blunt about the cost: multi-agent research systems use roughly 15x more tokens than a standard chat interaction, because every parallel sub-agent carries its own full context window, its own tool calls, and its own intermediate reasoning before anything gets compressed and handed back up.
Anthropic also ran the numbers on what actually predicts research quality. On their BrowseComp evaluation, token usage alone explained about 80% of the performance variance, ahead of tool-call count and model choice combined. In their words, this happens because "multi-agent architectures effectively scale token usage for tasks that exceed the limits of single agents." The implication for infrastructure: don't size your GPU cluster around how many concurrent users you expect. Size it around total token throughput across every sub-agent your supervisor is likely to spawn per report, because that's the number that actually consumes VRAM and compute.
Open Deep Research and GPT Researcher: Architecture and What They Need From You
Both projects are mature enough to run in production, and both are permissively licensed with no restrictions on self-hosting or commercial use. Here's what each one actually requires from your infrastructure.
Open Deep Research: Supervisor Delegation, MCP Support, Model-Agnostic Setup
Open Deep Research is LangChain's fully open-source agent, built on LangGraph, sitting at roughly 12,000 GitHub stars and 1,700 forks. It's MIT licensed. The pipeline runs in three phases: scope (clarify the request and produce a research brief), research (the supervisor delegates to parallel sub-agents), and write (a single-pass final report generation over everything the sub-agents returned).
What makes it worth self-hosting rather than treating as a LangChain-only tool is that it's genuinely model-agnostic. Every role in the pipeline routes through LangChain's init_chat_model() API, which means you can point the summarization step, the research agents, the compression step, and the final report generator at four different models if you want a cost-tiered setup, or at one self-hosted vLLM endpoint if you want to keep everything in-house. The framework requires models that support structured outputs and tool calling, which rules out some smaller open-weight models but is a non-issue for anything in the Qwen3, Llama, or GLM families at 7B and above.
Search-wise, Tavily is the default, but Open Deep Research also supports native web search from Anthropic and OpenAI, plus full Model Context Protocol compatibility. That last part matters if you're already running MCP servers for other agent tooling: our MCP orchestration and autoscaling guide covers wiring MCP servers into agent fleets running on GPU cloud, and the same pattern applies here. On Deep Research Bench, a leaderboard covering 100 PhD-level tasks across 22 academic fields, Open Deep Research's own submission (GPT-4.1-nano for summarization, GPT-4.1 for research and compression) landed a 0.4344 RACE score, good for #6 on the leaderboard, which puts it solidly in the same tier as commercial deep research products despite being free to run.
GPT Researcher: Planner-Executor-Publisher Pipeline, Local + Web Hybrid Research, Provider Flexibility
GPT Researcher, built by Assaf Elovic, is the more widely adopted of the two by star count: roughly 28,000 GitHub stars, Apache-2.0 licensed. Its planner-executor-publisher pipeline is simpler to reason about than Open Deep Research's supervisor model, which also makes it a bit easier to self-host if you want predictable, front-loaded resource usage rather than an adaptive supervisor that might decide to spawn more sub-agents mid-run.
The standout feature for self-hosters is provider flexibility. GPT Researcher supports over 100 LLM providers, including OpenAI, Anthropic, Groq, Llama 3, and Hugging Face-hosted models, so pointing it at a self-hosted vLLM endpoint is a first-class, documented path rather than a workaround. It also runs hybrid research: alongside live web search, it can research over local documents (PDF, CSV, Word, Markdown), which is genuinely useful if part of your research task involves internal company documents that should never leave your infrastructure. That local-document mode reuses the same retrieval infrastructure covered in our agentic RAG on GPU cloud guide, since under the hood it's an embedding-and-retrieval pipeline feeding into the same LLM.
On cost and speed, GPT Researcher's own documentation is worth knowing before you plan a deployment: a deep research run takes around 5 minutes to complete and costs roughly $0.40 using o3-mini at high reasoning effort, while a standard research task averages around 3 minutes and $0.10, and quick-search mode runs in 2 to 5 seconds. It also ranked #1 on Carnegie Mellon's DeepResearchGym benchmark across 1,000 complex queries in May 2025, ahead of both Perplexity Deep Research and OpenAI Deep Research on that evaluation. If you want a broader look at how research and coding agents get evaluated at scale, our benchmarking infrastructure guide covers the GPU setup for running SWE-bench, GAIA, and similar suites on your own hardware.
GPU and Model Requirements to Run One at Production Quality
The model you pick matters more here than in most agent workloads, because a deep research agent's output is judged on the quality of a long, cited, structured document, not on a single short answer.
Picking an Open-Weight Backbone for Long-Form, Cited Report Writing
Raw parameter count is not the variable to optimize for. Context length and instruction-following quality matter more, because a report-writing pass has to hold compressed findings from every sub-agent in context simultaneously and then follow a structured writing format without drifting. A model that's excellent at short factual Q&A but weak at sustained, structured long-form generation will produce reports that read like a list of disconnected facts rather than a coherent analysis.
In practice this points toward instruction-tuned models in the 32B-70B+ range with long native context windows (32K+) for the supervisor and writer roles, and smaller, faster 7B-8B models for the sub-agent research loop, where speed and tool-calling reliability matter more than writing polish since sub-agent output gets compressed before a human ever reads it. Our open-source LLM and VRAM guide has a fuller breakdown of which current open-weight models fit which VRAM budget if you're choosing a specific model rather than a size class.
VRAM Sizing by Model Tier
Here's how the two tiers break down in practice:
| Tier | Role | Example model class | VRAM (FP8) | Fits on |
|---|---|---|---|---|
| Fast/cheap sub-agent | Parallel research fan-out, tool calling | 7B-8B instruction-tuned | ~8-9 GB weights + 4-10 GB KV cache for 10-20 concurrent sub-agent calls | A100 40GB, L40S |
| Supervisor/writer | Task decomposition, final report synthesis | 32B-70B instruction-tuned | ~35-40 GB weights + 15-25 GB KV cache for long-context report writing | H100 80GB, H200 |
A minimal single-GPU setup runs one 32B-70B model for every role on a single H100 or H200. It's simpler to operate and cheaper in absolute GPU count, but every sub-agent call pays the latency and cost of a larger model even when the task is a simple tool-calling loop. A tiered setup splits sub-agent fan-out onto a cheaper, faster model and reserves the larger model for supervisor reasoning and the final write pass. The tiered split is what most production deployments converge on, because sub-agent calls vastly outnumber report-writing calls per research task, and running all of them on a 70B model wastes GPU-seconds on work a 7B model handles just as well.
Why Token Volume, Not Concurrency, Is the Real GPU Sizing Driver
Go back to Anthropic's finding: token usage alone explains roughly 80% of research quality variance, and multi-agent systems burn about 15x the tokens of a normal chat turn. That means the question to ask when sizing GPUs isn't "how many users will hit this at once," it's "how many sub-agent tool-calling turns does an average report generate, times how many reports run per day." A single user running one deep research report can generate more total tokens across sub-agents than 50 concurrent users chatting with a normal LLM assistant. Plan your --max-num-seqs and KV cache budget in vLLM around expected sub-agent fan-out per report, not around concurrent human sessions, or you'll under-provision a setup that looks fine on paper.
Deploying Open Deep Research or GPT Researcher on Spheron
Provisioning, Serving the Backbone With vLLM, Wiring Search Tools
The deployment shape is the same regardless of which framework you pick, because both talk to any OpenAI-compatible endpoint.
Start by provisioning the GPU. For a single-tier setup, an H100 on Spheron running a 32B-70B model in FP8 handles the full pipeline. For a tiered setup, put the writer model on an H200 GPU instance for the extra context headroom and run the sub-agent tier on a smaller Spheron A100 instance at a fraction of the hourly cost. Log in to app.spheron.ai, choose Ubuntu 22.04 with CUDA 12.4, and deploy. See Spheron's documentation for API-based provisioning if you want to script this as part of a CI pipeline.
Serve the backbone with vLLM:
docker run --gpus all --ipc=host -p 8000:8000 \
vllm/vllm-openai:latest \
--model <your-chosen-model> \
--quantization fp8 \
--max-model-len 32768 \
--gpu-memory-utilization 0.85Set --max-model-len generously. Report-writing prompts in both frameworks accumulate long context from compressed sub-agent findings, and truncating that context mid-report is a common cause of incomplete or incoherent output.
For Open Deep Research, clone the repo and configure each model role through LangChain's init_chat_model(), pointing the summarization, research, compression, and report-generation roles at your vLLM endpoint (or splitting them across your tiered GPUs). For GPT Researcher, clone the repo, set OPENAI_BASE_URL in your .env to the vLLM endpoint, and set FAST_LLM, SMART_LLM, and STRATEGIC_LLM to route each pipeline stage to the right backbone tier.
For search, Tavily is the path of least resistance for both frameworks and needs only an API key. If you want zero external search dependencies, self-host SearXNG next to your GPU instance and point either framework's retriever configuration at it instead. Open Deep Research's MCP support also means you can wire in custom internal search tools as MCP servers rather than relying only on public web search, which matters for research tasks that need to pull from internal knowledge bases alongside the open web.
Run a test report and watch nvidia-smi during the run. Utilization should spike during sub-agent fan-out, when multiple parallel tool-calling loops hit the LLM concurrently, and again during final report synthesis. If sub-agent calls appear to queue instead of running in parallel, raise vLLM's --max-num-seqs to allow more concurrent generation slots.
Cost of Self-Hosting vs Paying for Hosted Deep Research
What OpenAI's and Perplexity's Deep Research Actually Cost per Query
OpenAI's o3-deep-research is priced at $10 per million input tokens and $40 per million output tokens. For a realistic research task (roughly 50,000 input tokens, 20,000 output tokens, and 15 web searches), that works out to about $1.45 per query: $0.50 in input cost, $0.80 in output cost, and $0.15 in mandatory search fees billed at $10 per 1,000 calls, per tokencost.app's breakdown of the pricing. The mandatory search component can't be disabled on deep research models, so it adds up regardless of how efficient your prompting is.
The cheaper o4-mini-deep-research tier runs $2 per million input tokens and $8 per million output tokens, averaging around $0.41 per query for the same realistic scenario. That's the more relevant comparison point for most self-hosting decisions, since it's the tier most teams would actually default to for volume research.
Perplexity's pricing has its own layers. Perplexity Pro at $20 a month includes 20 Deep Research queries a day through the chat interface, and Perplexity Max at $200 a month gives unlimited Deep Research queries, but neither unlimited tier extends to API access. The underlying Sonar Deep Research API charges a $2/$8 base token rate plus $2 per million citation tokens, $3 per million reasoning tokens, and a $5-per-1,000-query search fee, landing a full query at roughly $0.41 or more depending on how deep the research goes.
Self-Hosted Cost per Report on Rented GPUs and Where the Breakeven Falls
Here's the math with live numbers rather than estimates. As of this writing, an H100 SXM5 on Spheron runs $2.54/hr on-demand, and an A100 80GB (SXM4) runs $1.69/hr on-demand. A single H100 handling every role in a research pipeline (supervisor, sub-agents, and writer, all on one FP8 32B-70B model) is the simplest cost model to reason about.
Assume a report takes roughly 5 minutes of GPU-bound compute time, in line with GPT Researcher's own documented deep-research runtime of about 5 minutes per run, extended somewhat by the longer multi-hop sub-agent fan-out typical of Open Deep Research's supervisor pattern. At Spheron's per-minute billing, that's about $0.21 in GPU time per report ($2.54/hr ÷ 60 × 5 minutes). At 500 reports a month, that's roughly $105.83 in GPU spend, against $205 for the same volume on o4-mini-deep-research or $725 on o3-deep-research. At 5,000 reports a month, self-hosted GPU cost scales to roughly $1,058, against $2,050 (o4-mini) or $7,250 (o3) hosted.
The gap widens with volume because per-minute GPU billing doesn't carry the per-token and per-search-call markup that hosted APIs charge on every request. It narrows, and can reverse, at low volume, because a hosted API has effectively zero fixed cost while a dedicated GPU instance has a floor cost even with per-minute billing if you're keeping a model warm to avoid multi-minute cold-start delays on the first request of the day.
Pricing fluctuates based on GPU availability. The prices above are current as of 11 Jul 2026 and may have changed. Check current GPU pricing → for live rates.
When Hosted Still Wins (Low Volume, No Ops Capacity)
Self-hosting isn't the right call for everyone. If you're running a handful of research reports a week rather than hundreds a month, the fixed cost of keeping a GPU instance warm and the engineering time to maintain a vLLM deployment, monitor sub-agent token usage, and keep search integrations working will cost more in practice than just paying per query. Teams without existing DevOps or MLOps capacity should also weigh the maintenance burden honestly: self-hosting means you own model upgrades, prompt tuning as new open-weight models ship, and search tool reliability, none of which OpenAI or Perplexity charge you extra to handle on their side.
Anthropic frames the entire multi-agent research pattern as economically justified only for high-value research, legal due diligence, competitive intelligence, literature review, where the answer is worth paying for the token multiplier. That framing applies just as much to the self-host-versus-hosted decision. If your research tasks are occasional and low-stakes, a hosted Deep Research subscription is the simpler, cheaper choice. If you're running research at team or product scale, with hundreds to thousands of reports a month, the GPU math above is where self-hosting starts winning decisively.
Both Open Deep Research and GPT Researcher are one clone and one vLLM endpoint away from running on your own infrastructure. Spheron's per-minute billing means you only pay for the GPU-minutes each report actually uses, not a flat hourly rate whether the model is generating or idle.
Check H100 availability → | Rent H200 GPU → | View all GPU pricing →
Quick Setup Guide
List the roles your agent needs: sub-agent research calls, summary/compression, and final report writing. Estimate tokens per role using Anthropic's own scaling guidance for this architecture as a starting point: 3-10 tool calls for a simple fact-finding subagent, 10-15 calls per subagent for direct comparisons, and 10 or more subagents for complex, multi-part research. Multiply by your expected report volume, not by concurrent users, since sub-agent fan-out multiplies token load independent of how many people are waiting on a report.
Log in to app.spheron.ai and select a GPU sized to your chosen backbone: an H100 SXM5 (80GB) for a single 32B-70B FP8 model handling all roles, or an H200 (141GB) if you want headroom for a larger writer model plus KV cache for several concurrent research sessions. Choose Ubuntu 22.04 with CUDA 12.4 and deploy. Verify GPU visibility with nvidia-smi over SSH.
Launch an OpenAI-compatible endpoint: docker run --gpus all --ipc=host -p 8000:8000 vllm/vllm-openai:latest --model <your-chosen-model> --quantization fp8 --max-model-len 32768 --gpu-memory-utilization 0.85. Set --max-model-len generously since report-writing prompts accumulate long context from compressed sub-agent findings. Confirm the endpoint with curl http://localhost:8000/v1/models.
For Open Deep Research, clone the repo, set OPENAI_API_BASE (or use LangChain's init_chat_model with a custom provider) to your vLLM URL, and configure separate model variables for the summarization, research, compression, and final-report roles if you're running a tiered setup. For GPT Researcher, clone the repo, set OPENAI_BASE_URL in your .env to the vLLM endpoint, and set FAST_LLM / SMART_LLM / STRATEGIC_LLM to route each pipeline stage to the right backbone tier.
Open Deep Research defaults to the Tavily API but also supports native Anthropic and OpenAI web search plus any MCP server. GPT Researcher supports Tavily, SearXNG (self-hosted, no API key needed), and several other retrievers via its RETRIEVER environment variable. If you want zero external search dependencies, self-host SearXNG alongside your GPU instance and point either framework at it.
Trigger a research task through either framework's CLI or API and watch nvidia-smi during the run. You should see utilization spike during sub-agent fan-out (multiple parallel tool-calling loops hitting the LLM concurrently) and again during final report synthesis. If sub-agent calls queue up and utilization stays low, increase --max-num-seqs on the vLLM server to allow more concurrent generation slots.
Frequently Asked Questions
Open Deep Research is LangChain's MIT-licensed agent built on LangGraph. A supervisor model decomposes a research brief and delegates to parallel sub-agents, each with an isolated context window, then a separate model compresses findings and writes the final report. GPT Researcher is Apache-2.0 licensed and uses a simpler planner-executor-publisher pipeline: a planner generates research questions, executor agents gather sources for each one, and a publisher assembles the cited report. Open Deep Research leans harder into parallel sub-agent fan-out and MCP tool integration; GPT Researcher leans into provider flexibility (100+ LLM providers) and hybrid local-document plus web research.
It depends on which roles you run locally. A minimal setup uses one open-weight model for everything: a 32B-70B instruction-tuned model in FP8 fits on a single H100 (80GB) or H200 (141GB) and can serve as supervisor, sub-agent, and report writer. A tiered setup splits the work: a 7B-8B model for sub-agent fan-out (fits on an A100 40GB or L40S with room for concurrent KV cache) plus a 70B+ model for supervisor reasoning and final report synthesis (H100 or H200). The tiered setup costs more in total GPU-hours but produces noticeably better long-form report quality from the writer model.
At meaningful volume, yes. OpenAI's o4-mini-deep-research runs about $0.41 per query and o3-deep-research about $1.45 per query for a realistic 50K-input/20K-output/15-search task, per tokencost.app's pricing breakdown. Perplexity's Sonar Deep Research API layers citation and reasoning tokens plus a $5-per-1,000-query search fee on top of its base rate, landing around $0.41 or more per query depending on depth. A single self-hosted H100 SXM5 at Spheron's on-demand rate handles hundreds of reports for what one day of moderate hosted API usage costs, because GPU rental is billed per minute of actual use, not per token and per search call. Below a few hundred reports a month, the math favors hosted; above a few thousand, self-hosting usually wins.
Anthropic's own multi-agent research system, the architecture both open-source projects mirror, uses roughly 15x more tokens than a single chat interaction because every parallel sub-agent carries its own full context window, tool calls, and intermediate reasoning. Anthropic found that token usage alone explains about 80% of the performance variance on their BrowseComp evaluation, ahead of tool-call count and model choice combined. That means GPU and inference budget for a deep research agent should be planned around total token throughput across all sub-agents, not around how many concurrent users you expect.