SGLang Benchmark Agent Sandbox: vLLM vs SGLang From Chat (2026)

An SGLang benchmark agent sandbox run turns "trust someone else's numbers" into something you can watch happen: a chat agent provisions the GPU itself, serves the same model under both vLLM and SGLang, runs each engine's own official load-testing tool, and reports tokens per second and time to first token back in the same conversation. Spheron's Agent GPU Sandbox lists exactly this, benchmarking vLLM or SGLang from chat, as a named use case on its own feature page.

TL;DR: What Happens in an SGLang Benchmark Agent Sandbox Run?

Yes: an SGLang benchmark agent sandbox run has Spheron's Agent GPU Sandbox provision 2x H100, install vLLM and SGLang side by side, and run each engine's own benchmark tool as one background job under a 24-hour ceiling, so you read tokens/sec and TTFT for both engines in a single chat thread instead of two manual setups.

• Prompt: Spheron's documented baseline is "Benchmark vLLM serving Qwen3 32B on 2x H100 and tell me the tokens per second," extended here to run the same model under SGLang too.
• Hardware: the sandbox's feature page suggests 2x H100 for this comparison.
• Commands: vLLM's official tool is vllm bench serve; SGLang's is python3 -m sglang.bench_serving, and both report throughput and TTFT, not one number alone.
• Sanity check: Spheron's own fleet tests on 8xB200 put SGLang-served GLM-5.3 at 117 tok/s per user at 32 concurrent users with a 0.90-second TTFT, as of Oct 2026, a useful range to compare an agent's reported numbers against.

The Prompt and the GPU the Agent Picked

Spheron's documentation gives the baseline prompt as "Benchmark vLLM serving Qwen3 32B on 2x H100 and tell me the tokens per second." A two-engine comparison adds the second engine to the same instruction, against the same checkpoint and GPU count, unlike most benchmark posts online, which are a script dump against whatever GPU the author had, for one engine at a time. Spheron's feature page names the two-engine version directly: "benchmark vLLM or SGLang on one GPU or several and get tokens per second and latency in your chat," suggesting 2x H100.

Two H100 instances give a 32B dense model enough VRAM headroom under both engines at once without a B200 pair the comparison doesn't need. For a larger MoE checkpoint, name the bigger tier in the prompt instead.

Setting Up the Agent Before You Send the Prompt

Running this hands-off starts with four things that have nothing to do with vLLM or SGLang at all, and skipping any one of them is the usual reason a first attempt stalls before the agent ever touches a GPU.

1. Link the AI connection. Spheron's AI connections page is where you pair the account with the agent that's going to run the sweep, whether that's Claude, Cursor, Codex CLI, or any other MCP client. This is the step that lets the agent see GPU offers and build a deploy preview in the first place.
2. Save an SSH key. Add at least one key under Settings > SSH Keys before you deploy anything. That key works on the instance before, during, and after the agent's sweep, so you can log in and tail a log yourself without waiting for the agent to hand the machine back.
3. Load account credit. The agent can pick a GPU, price it out, and show you a deploy preview against an empty balance, but it can't actually provision the instance until there's credit behind it.
4. Set a spend limit on the connection. Qwen3 32B under vLLM and SGLang across a warm-up pass and four concurrency levels is a multi-hour job on a two-GPU instance. A spend limit caps what the agent can launch without you watching the clock, which is the entire point of handing a benchmark sweep to an agent instead of running it yourself.

Once those four are in place, the prompt does the rest. Spheron's documented baseline, "Benchmark vLLM serving Qwen3 32B on 2x H100 and tell me the tokens per second," names the model, the engine, and the hardware in one line, and extending it to a second engine is just adding the comparison: "and also benchmark it under SGLang, then compare the two engines." The agent resolves the GPU, assembles the deploy preview, and waits on your one approval before it provisions anything, which is the only manual step in the whole run.

What You Do While the Agent Handles the Rest

From that single approval onward, the process is built to need no further input from you until the numbers come back. The agent's shell access runs with sudo on the instance, so it installs both engines' dependencies, pulls the checkpoint, starts and stops each serving process, and launches both benchmark tools without asking again at each step. Anything that would run past the 10-minute foreground limit, including the full vLLM-then-SGLang sequence described above, goes out as a background job the agent polls instead of a command that blocks the chat for however long the sweep takes.

A few things are worth knowing before you walk away from the chat entirely:

• Chat output has a ceiling. Only the first 1MB of any command's output returns to the conversation; the rest stays on the machine under /var/log/spheron-agent/, where the agent can read it back on request. A four-concurrency sweep across two engines produces more raw output than that, so expect a summary of the throughput and TTFT numbers in chat with the full logs sitting on the box rather than pasted in full.
• Your SSH access doesn't pause for the agent's. The key you saved earlier works on the same instance the whole time, so nothing stops you from logging in mid-sweep to check nvidia-smi or tail a log while the agent is still running the SGLang half of the comparison.
• Nothing is reachable from outside. The sandbox's only network path is the agent's own outbound HTTPS connection; no inbound port opens on the instance, and only the account that deployed it can drive its sandbox, so a sweep you kick off isn't something a teammate can quietly redirect.
• Closing it out is also a prompt, not a console tab. Once the numbers are in, you can ask the agent to stop the instance, which keeps the disk and bills only storage, useful if you want to rerun the same sweep later against a newer SGLang release without re-pulling the weights. Asking it to terminate instead, once the minimum runtime has elapsed, ends billing entirely.

The upshot is that the only two moments you're actually needed are approving the deploy preview at the start and reading the tokens-per-second and TTFT numbers at the end. Everything in between, including the parts that would otherwise mean babysitting two separate engine installs and a multi-hour load test, is the agent's job to run and yours to walk away from.

What the Agent Actually Ran and Logged

Once the deploy preview is approved, the agent installs both stacks, then pulls the weights with huggingface-cli instead of through chat. The Agent GPU Sandbox docs cap file transfer into the VM at 5MB per file, well under any real checkpoint, set a 10-minute default timeout on a foreground command, and allow up to 24 hours on a background one. It serves the model first under vLLM with vllm serve, then runs vllm bench serve against that endpoint, reporting TTFT, TPOT, and ITL per request. It then stops that process, starts sglang.launch_server for the same model, and runs python3 -m sglang.bench_serving --backend sglang --max-concurrency <N> --num-prompts <N> --random-input-len <N> --random-output-len <N> at matching concurrency levels. None of that fits inside the 10-minute foreground limit, so the whole sequence runs as a single background job instead.

Warm-up isn't optional, either. Benchmarking a freshly booted engine measures its kernels compiling, not its real throughput: in Spheron's own fleet benchmarking on 8xB200 on 15 Oct 2026, a GLM-5.3-Flash sweep right after boot measured 34 tok/s per user at 32 concurrent users with a 40-second first token, about a quarter of the 129 tok/s and 0.84-second TTFT it holds once its linear-attention kernels finish compiling. An agent that times the sweep the moment an engine reports "ready" logs that same slow, cold-start result, which is why the run needs a throwaway warm-up pass first.

Reading the Results: Throughput and TTFT Side by Side

Both tools report throughput and latency in slightly different shapes. A fair vLLM-vs-SGLang read uses both sides of the table, not whichever metric flatters the engine you already prefer.

There's no agent-run Qwen3 32B figure here; inventing one would be worse than leaving it blank. What's genuinely available as a sanity check is Spheron's own fleet methodology: a speed sweep measuring P50 output tokens per second per user and TTFT across 1, 8, 16, and 32 concurrent users, taken warm, the same shape this walkthrough's agent runs. On 8xB200, SGLang serving GLM-5.3 measured 324 tok/s for one user, 158 at 8 users, 142 at 16, and 117 at 32, with a 0.90-second TTFT, as of Oct 2026; vLLM serving DeepSeek V4 Flash on the same node class measured 272 tok/s for one user, 170 at 8 users, 140 at 16, and 116 at 32, with a 0.64-second TTFT. Different models, same hardware class, but the shape is the useful part: tok/s per user drops as concurrency climbs, TTFT stays under a second once warm, and the engine ahead at one user isn't automatically ahead at 32. A TTFT in the tens of seconds is the warm-up gotcha above, not a real engine difference.

SGLang's RadixAttention wins on prefix-heavy traffic like multi-turn agents and RAG, a decision table the TGI migration guide lays out in full, and SGLang's newer Decisions API is a reminder vLLM doesn't mirror every SGLang-only endpoint.

Cost and Runtime for a Full SGLang Benchmark Agent Sandbox Sweep

The sandbox itself carries no separate fee. You pay the GPU instance's normal hourly rate for however long it runs, the same rate as renting it directly. Two Spheron H100s on-demand run $5.28/hr per hour total ($2.64/hr each) as of 02 Nov 2026, for the hours the warm-up and both sweeps take.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 02 Nov 2026; other figures reflect benchmarks recorded on their stated dates and may have changed. Check current GPU pricing → for live rates.

Every deploy, Stop, Start, and terminate shows that price preview and needs explicit approval, so a multi-concurrency sweep can't quietly run past a budget you didn't see. It's also a multi-minute GPU workstation, not a sub-second code sandbox: Spheron's AI agent code execution sandbox guide covers the CPU-only tools, E2B and Daytona among them, built for instant iteration instead. Once you've picked an engine, the next benchmark worth running is an agent-eval one: the tool-calling benchmark guide and the LoRA fine-tuning walkthrough both run on the same sandbox mechanics.

Benchmarking vLLM against SGLang only answers the question if the agent running it can actually get its hands on a second GPU and both engines without you doing the setup yourself, which is what the Agent GPU Sandbox is built for.

Connect your agent to the Agent GPU Sandbox →