Deploy OpenClaw on GPU Cloud: Self-Host the Open-Source Agentic Assistant With Your Own LLM Backend (2026 Guide)

OpenClaw hit 370K+ GitHub stars faster than almost any open-source AI project in 2026. It's a fully self-hosted agentic assistant with 50+ built-in tool integrations, MCP compatibility, and a design that puts the LLM backend entirely under your control. The appeal is obvious: the same kind of capability people pay for in hosted agents, running on your own infrastructure, calling your own model.

The catch most teams hit: OpenClaw defaults to the OpenAI API. At agent-loop token volumes, the per-token bill grows fast. More importantly, every prompt and tool result goes through a third-party endpoint. For teams with data sensitivity requirements or predictable cost needs, the default setup doesn't work. Teams who have already moved to self-hosted frontends like Open WebUI or LibreChat face the same question with OpenClaw: how do you point it at your own model instead of the vendor API?

This guide covers the private GPU backend path. You run a vLLM inference server on a Spheron GPU instance, point OpenClaw at it, and keep everything on your own network. If you want context on setting up an OpenAI-compatible private API endpoint first, that guide covers the vLLM server setup in isolation.

What Is OpenClaw and Why It Went Viral

OpenClaw is structured around three components: a controller process, a tool registry, and an LLM backend. The controller manages the agentic loop. The tool registry holds the 50+ integrations: browser automation, code execution, file system access, web search, email, calendar, Slack, GitHub, and more. The LLM backend is the model that reasons over tool results and decides what to do next. The backend is pluggable: point it at any OpenAI-compatible endpoint and it works.

That three-component split is what makes self-hosting viable without forking the project. The controller and tool registry are CPU processes. The LLM backend is just an HTTP endpoint. Swap the OpenAI API URL for your vLLM server address and nothing else changes in the stack.

The viral growth came from a few things stacking up. MCP compatibility means you can connect OpenClaw to any existing MCP tool server, not just the built-in 50. The UI is polished in a way most open-source agent projects aren't. And the fully self-hosted story resonated with teams that had already moved away from ChatGPT for data reasons but still wanted an assistant that could actually do things, not just answer questions.

For data privacy, the self-hosted LLM path is what closes the loop. Tool calls that connect to Slack or GitHub still send data to those services, since that's unavoidable if you want those integrations. But the LLM reasoning, the prompts, the accumulated context in the agentic loop, none of that leaves your network if the backend is a vLLM server on your own GPU node.

Architecture: Two-Node Setup

The production setup uses two nodes:

The communication path is: OpenClaw UI -> OpenClaw Controller -> vLLM HTTP endpoint on GPU node. The controller sends LLM requests to http://<inference-node-ip>:8000/v1 (OpenAI-compatible). Only the inference node needs GPU access. The controller node runs OpenClaw itself plus any local MCP servers you deploy (a local Chroma instance for RAG, a code execution sandbox, etc.).

For development or small-team testing, you can run both on the same GPU node. The separation matters when you want to scale the inference node independently, or when you want the CPU/memory bill for the controller to stay flat as GPU capacity scales up. A second scenario: if you run multiple OpenClaw instances pointing at the same vLLM endpoint, the controller nodes are cheap horizontal scale while the GPU stays singular. This two-node split is the same pattern as deploying OpenHands on GPU cloud, where a separate agent controller and GPU inference backend give you the same scaling flexibility.

Picking the LLM: Model Selection for Agentic Tool-Calling

The LLM you choose determines VRAM requirements, tool-call accuracy, and latency per step. In an agentic loop, each tool call triggers an LLM request: read the tool result, decide what to do next, emit a function call or a final answer. At 10-20 steps per task, time to first token compounds. H100 prefill speed is noticeably faster than A100 at the same model size, which shows up as task duration at production volumes.

Qwen3-32B is the default recommendation for OpenClaw. The Qwen3 series has native structured output and function-calling support, and with --tool-call-parser hermes in vLLM the tool calls parse reliably. OpenClaw's 50+ tool integrations mean you will hit edge-case schema patterns regularly. Models without mature function-calling support start failing on those edge cases.

The --tool-call-parser flag is model-family-specific. Qwen3 uses hermes. Llama 3.x uses llama3_json. Mistral uses mistral. Using the wrong parser for your model causes silent tool-call failures: the agent receives what looks like a valid response but the function call is malformed, and it spins without making progress. Always match the parser to the model family.

For scaling patterns when you outgrow a single GPU, scaling AI agent fleets on GPU cloud covers multi-node orchestration for agent workloads.

GPU Sizing and Pricing

Pricing fluctuates based on GPU availability. The prices above are based on 16 Jun 2026 and may have changed. Check current GPU pricing → for live rates.

For a single team running OpenClaw all day, one H100 SXM5 at $3.98/hr runs to roughly $95/day. That covers continuous operation, not just hours you're actively using it. For intermittent use (a few hours of active sessions per day), spot pricing at $2.91/hr brings the daily cost under $35. vLLM idles gracefully between requests, so you're paying for the GPU reservation, not per-call compute.

Step-by-Step Deployment

Step 1: Provision GPU and controller nodes

Log into app.spheron.ai and provision an H100 SXM5 80GB instance as the inference node. For the model weights, attach at least 100 GB persistent storage: Qwen3-32B at BF16 is ~65 GB of model weights plus cache overhead. For the controller node, a standard CPU instance with 8-16 vCPU and 32 GB RAM is sufficient.

For H100 SXM5 on Spheron, use the SXM5 variant specifically if you plan to use MIG partitioning for concurrent sessions later. PCIe variants don't support MIG.

Step 2: Install vLLM and download model weights

On the inference node:

pip install 'vllm>=0.8.0' huggingface_hub hf_transfer
export HF_TOKEN=your_hf_token
export HF_HUB_ENABLE_HF_TRANSFER=1

# For Qwen3-32B
huggingface-cli download Qwen/Qwen3-32B

# For Llama 3.3 70B (FP8 budget option)
huggingface-cli download meta-llama/Llama-3.3-70B-Instruct

The HF_HUB_ENABLE_HF_TRANSFER=1 flag switches to a faster transfer protocol that cuts download time significantly on high-bandwidth GPU nodes.

Step 3: Launch the vLLM inference server

For Qwen3-32B at BF16 on H100 SXM5:

vllm serve Qwen/Qwen3-32B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --port 8000 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

For Llama 3.3 70B at FP8 on A100 80GB:

vllm serve meta-llama/Llama-3.3-70B-Instruct \
  --quantization fp8 \
  --max-model-len 32768 \
  --port 8000 \
  --enable-auto-tool-choice \
  --tool-call-parser llama3_json

The --enable-auto-tool-choice flag is what makes vLLM emit proper function-call formatted responses instead of plain text. Without it, OpenClaw receives text it can't parse as a tool call, and the agentic loop breaks silently. The --tool-call-parser must match the model family: hermes for Qwen3, llama3_json for Llama 3.x, mistral for Mistral family.

vLLM binds to all interfaces by default. For production, restrict it:

vllm serve Qwen/Qwen3-32B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --port 8000 \
  --host <internal-network-ip> \
  --api-key your-strong-random-key \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Step 4: Install and configure OpenClaw

Install OpenClaw on the controller node:

npm install -g openclaw@latest
openclaw onboard

The onboard command walks through backend configuration interactively. When prompted for the LLM provider, select the OpenAI-compatible option and enter your vLLM server URL (http://<gpu-node-ip>:8000/v1) and the API key you set with VLLM_API_KEY. OpenClaw stores configuration in a JSON file. The key setting is agent.model, which uses a <provider>/<model-id> format. For a vLLM backend with Qwen3-32B, that value is openai-compatible/Qwen/Qwen3-32B.

Before running a session, verify the vLLM endpoint is reachable from the controller node:

curl -s -o /dev/null -w "%{http_code}" http://<gpu-node-ip>:8000/health
# Should return 200

If that returns anything other than 200, the inference node's firewall or security group is blocking port 8000 from the controller.

Step 5: Wire up MCP tools without external data leakage

OpenClaw's 50+ built-in tools vary in their data exposure. Browser automation, GitHub integration, and email tools send data to those external services, which is expected behavior. The tools to audit are any that use hosted AI backends behind the scenes: cloud-based RAG services, hosted embedding APIs, third-party summarization tools. Those send your agent's context to external endpoints.

For private RAG, run Chroma or Qdrant locally on the controller node with a local embedding model:

docker run -d -p 8001:8000 chromadb/chroma

Wire it as a local MCP server in OpenClaw's MCP config. The GPU-accelerated MCP server deployment guide covers how to wrap self-hosted retrieval backends as MCP tools with proper session handling for multi-turn agent conversations.

For the code execution tool, OpenClaw supports Docker-based sandboxing. Configure it to run sandboxes with network disabled if your tasks don't need outbound access:

sandbox:
  network_disabled: true
  runtime: docker

Step 6: Secure the vLLM endpoint

The VLLM_API_KEY environment variable locks the vLLM endpoint behind token authentication:

export VLLM_API_KEY=your-strong-random-key
vllm serve Qwen/Qwen3-32B --dtype bfloat16 --port 8000 ...

Match the api_key in OpenClaw's config to this value. In production, bind vLLM to the internal network interface only (--host <internal-ip>) so the inference port is not reachable from the public internet. Only the OpenClaw controller node needs HTTP access to port 8000 on the inference node.

For the security group or firewall rules: allow TCP 8000 inbound on the inference node only from the controller node's internal IP. Block it from everything else.

Cost: Self-Hosted GPU Backend vs Hosted API Per-Token Pricing

The cost comparison depends on your daily output token volume. OpenClaw in active use generates high token counts: each tool-call step sends the accumulated context plus tool results back to the model. A single 20-step agent task can easily produce 5,000-10,000 output tokens across all the intermediate reasoning calls.

GPT-4o output pricing at $10/1M tokens (as of mid-2026). H100 costs are daily flat rate from Spheron live pricing.

The GPU cost is fixed whether you use it or not. At low token volumes (under 1-2M output tokens/day), the API is cheaper in direct compute terms. At higher volumes, the GPU wins. The break-even varies, but for a small team running OpenClaw actively, 2M+ daily output tokens is typical once it's actually deployed and used for real work.

What this table doesn't show: the GPU also handles MCP tool-result summarization calls. When OpenClaw uses a retrieval tool, it sends the retrieved chunks back to the model to reason over them. Those are invisible in per-token API billing but very visible at agent-loop volumes. The GPU absorbs all of them at the same flat daily rate.

Scaling for Teams

One vLLM instance handles concurrent OpenClaw sessions. The default --max-num-seqs in vLLM is 256, which is more than enough for small team usage. You can lower it to control maximum concurrency if you want to limit KV cache usage:

vllm serve Qwen/Qwen3-32B --max-num-seqs 32 ...

For 5-10 concurrent active users on Qwen3-32B, one H100 SXM5 handles the load comfortably. Each user's agentic session generates sequential LLM calls (one step at a time), not parallel ones, so the effective concurrency at any moment is much lower than the number of simultaneous sessions.

For teams above 10 concurrent active users: add a second H100 with tensor parallelism. Launch vLLM with --tensor-parallel-size 2 across two GPUs and it spreads the model weights. Alternatively, run two separate vLLM instances and load-balance with nginx upstream:

upstream vllm_pool {
    server gpu-node-1:8000;
    server gpu-node-2:8000;
}

Each OpenClaw controller points to the nginx upstream instead of a single node. The vLLM instances don't share KV cache, so session affinity doesn't matter for stateless tool-call patterns.

What to Watch in Logs

Three things break silently in vLLM-backed OpenClaw deployments:

Tool-call parsing failures. vLLM logs include the raw model output before parsing. If you see tool_call_parser error in the vLLM log, the model output didn't match the expected function-call format. The most common cause: using the wrong --tool-call-parser for the model family. Check the vLLM log at debug level to see the raw output.

Context window exhaustion in long loops. If an agentic task runs many steps, the accumulated context grows. When it exceeds --max-model-len, vLLM truncates or rejects the request. The symptom: OpenClaw tasks that succeed on short runs fail on complex multi-step tasks. Fix: increase --max-model-len (costs more KV cache VRAM) or implement context compression in OpenClaw's settings to prune old tool results from the context.

Queue depth under concurrency. The vLLM metrics endpoint at /metrics exposes vllm:num_requests_waiting. If this stays above 5 for more than 30 seconds, your GPU is undersized for the current concurrency. Add a second node or reduce the max number of concurrent sessions.

OpenClaw needs a GPU that stays up, not a serverless endpoint that cold-starts mid-agent-loop. Spheron H100 and A100 instances run on-demand with per-minute billing, so there's no idle waste between sessions.

H100 SXM5 on Spheron → | View all GPU pricing → | Get started →