SkyRL is the reinforcement learning library NovaSky-AI and Berkeley's Sky Computing Lab built for long-horizon, multi-turn agents, and its own example scripts spell out exactly what training a coding agent on it costs in GPUs: one 8x H100 node for an 8B model, two for a 30B one. This guide walks through that path end to end: what SkyRL's pieces actually do, the GPU counts verified against its own launch scripts, the SWE-Gym setup, the config values in the real launch command, the three failure modes documented in SkyRL's own README that burn GPU hours without you noticing, and what the run costs on an H100 node.
TL;DR: What Does It Take to Train a Coding Agent With SkyRL on GPU Cloud?
Training a coding agent with SkyRL on SWE-Gym needs a full node of 8x H100 GPUs for an 8B model, and two nodes (16x H100) for a 30B model, per SkyRL's own launch scripts.
- GPU count: SkyRL's mini_swe_agent example needs 1 node of 8x H100 for Qwen3-8B and 2 nodes of 8x H100 for Qwen3-Coder-30B, per the NovaSky-AI repo.
- Dataset: SWE-Gym supplies 2.4K real coding tasks from 11 Python repositories, plus a 234-instance Lite split for faster iteration.
- Stack: SkyRL runs GRPO through FSDP with 4 vLLM inference engines per node, checkpointing every 10 steps and logging to Weights & Biases.
- Cost: Spheron prices 8x H100 SXM5 nodes at $2.65/hr per GPU on-demand as of 16 Nov 2026; check current H100 pricing before committing to a multi-hour run.
- Failure modes: context-length overflow, all-zero reward batches, and an oversized-git-patch error are the three things SkyRL's own docs flag as silent GPU-hour burners.
What SkyRL Is (and Why It's Built for Long-Horizon Coding Agents)
Most RL-for-LLM frameworks grew up on single-turn chat: generate one completion, score it, update the policy. A coding agent doesn't work that way. It reads a repository, edits files, runs a test suite, reads the failure, edits again, and only gets its reward once that whole loop finishes, sometimes dozens of tool calls later. NovaSky-AI's SkyRL repository exists because that mismatch is structural. As the project's own page puts it, "most existing RL frameworks are optimized for tasks that involve stateless interactions over short horizons."
SkyRL splits the problem into four pieces. skyrl-train is the training framework itself: the FSDP trainer, the GRPO loss, the vLLM-backed rollout workers. skyrl-gym is a set of Gymnasium-API tool-use environments covering math, coding, search, and SQL, the same API surface as the Gymnasium-based tooling covered in our guide to RL environments on GPU cloud. skyrl-agent wraps a multi-step tool-use loop around that environment interface, and skyrl-tx is a Tinker-API-compatible backend for teams who'd rather drive SkyRL through that interface.
For a coding agent specifically, the piece that matters is the combination of skyrl-agent and skyrl-gym: one gives you the agent loop (read file, edit, run tests, read result, repeat), the other gives you the sandboxed environment that loop runs inside. The published example that wires both of these to a real coding benchmark is mini_swe_agent, which integrates the lightweight Mini-SWE-Agent framework with SkyRL and trains against SWE-Gym. That's the example this post follows, and it's also the clearest contrast with agent reinforcement fine-tuning on tool-calling trajectories: same GRPO-style mechanics, but the reward here comes from a real test suite instead of a verifiable API response.
SkyRL GPU Requirements: Qwen3-8B vs Qwen3-Coder-30B on SWE-Gym
SkyRL's own mini_swe_agent launch scripts are explicit about hardware, so you don't have to guess at VRAM math from first principles:
SkyRL's own launch scripts document hardware requirements explicitly. Neither is a theoretical minimum you could squeeze down with more aggressive offloading; they're the counts the maintainers actually ran and published scripts for.
That 8-GPU floor for even an 8B model surprises people coming from single-GPU SFT or LoRA fine-tuning. SkyRL isn't just holding one model in memory, it's running a trainer and multiple vLLM inference engines side by side, generating rollouts and updating weights in the same job. Coding and tool-use RL at the 7B-30B range consistently wants a full node, not a fraction of one.
If your target model sits above 30B, or you want to push past a single node, Spheron's H100 page lists on-demand and spot pricing for individual nodes, but multi-node clusters beyond a single 8-GPU box fall into its Custom Clusters tier, which covers 8 to 512+ GPUs and needs a quote with a 24-48 hour turnaround rather than an instant self-serve launch. Confirm that lead time before you schedule a 16-GPU SWE-Gym run.
Setting Up SWE-Gym and the Mini-SWE-Agent Environment
Start with the base install, because SkyRL is picky about versions in ways that are easy to miss. The documented requirements are CUDA 13.0 with NVIDIA driver r580 or newer, Python 3.12, uv for dependency management, and Ray, with 2.57.0 suggested specifically (2.44.0 and up is supported generally, but 2.47.0 and 2.47.1 are called out as not recommended). The simplest path to all of that at once is the maintained Docker image, novaskyai/skyrl-train-ray-2.57.0-py3.12-cu13.0, run with --runtime=nvidia --gpus all so every container on the node sees the full GPU set.
Once the environment is up, the mini_swe_agent example's preprocessing script, examples/mini_swe_agent/preprocess_swegym.py, pulls and formats the training split from SumanthRH/SWE-Gym-Subset. SWE-Gym itself is worth understanding before you point a training run at it: it's 2.4K real tasks drawn from 11 Python repositories, each shipped with its repository context and an executable, test-verified environment, plus a 234-instance Lite split for faster iteration while you're still debugging your config.
One setup step is easy to skip and shouldn't be: SWE-Gym's own README flags that its environments contain a git shortcut exploit similar to a documented SWE-bench issue, and it ships a cleanup script specifically to close that hole before training starts. Skip it and you risk an agent that learns to game the git state instead of fixing the bug, which looks like a win in your reward curve and is really a silent failure.
The environment backend for rollouts defaults to Podman, configurable in examples/mini_swe_agent/swebench.yaml if your infrastructure prefers Docker or another container runtime. Each rollout spins up an isolated container, applies the agent's generated patch, and runs the task's real test suite inside it, which is also where one of the three failure modes below comes from.
The Launch Command and Config You'll Actually Run
The 8B launch script, run_mini_swe_8B.sh, is the one to read line by line before you run it, because the values in it are the actual knobs that determine both training stability and GPU utilization. Pulled straight from the script in the NovaSky-AI repo:
- Algorithm: GRPO with a KL loss term against the reference policy.
- Training strategy: FSDP, with training and generation colocated on the same GPUs and a sequence parallel size of 2.
- Rollout engines: 4 vLLM inference engines, each at tensor-parallel size 2.
- Batch shape: a train batch size of 16, with a GRPO group size (
n_samples_per_prompt) of 4 rollouts per prompt. - Sequence limits: max prompt length and max generation length both set to 4096 tokens.
- Optimization: learning rate of 1e-6, run for 20 epochs.
- Checkpointing and logging: evaluation every 5 steps, checkpoints every 10 steps written to
~/ckpts/llm_mini_swe, metrics logged to Weights & Biases under the project namemini_swe.
If any of that feels like a lot to validate blind, it is, which is why SkyRL ships a much smaller sanity check first: the GSM8K GRPO quickstart trains Qwen2.5-1.5B-Instruct on just 4 GPUs with 4 vLLM engines at tensor-parallel size 1, launched with bash examples/train/gsm8k/run_gsm8k.sh. Run that first on whatever node you're planning to use. It confirms your Ray cluster, CUDA version, and vLLM engines are actually talking to each other before you commit a full 8x H100 node to a multi-hour SWE-Gym job and find out an hour in that the container runtime wasn't configured right.
What to Watch in the Logs (and the Failure Modes That Burn GPU Hours)
SkyRL's own mini_swe_agent README names three concrete failure patterns, and all three show up in the logs before they show up as a wasted bill:
- Context-length overflow. Multi-turn coding trajectories accumulate tokens fast: every tool call, every file read, every test output adds to the prompt. If a rollout's accumulated context exceeds your configured max length mid-trajectory, that step errors out rather than truncating gracefully.
- All-zero rewards. If an entire batch of rollouts comes back with zero reward, it isn't the agent being consistently bad, it's usually a signal that the task mix handed to this batch is too hard for the current policy to solve at all, and the batch is contributing no training signal while still consuming a full GRPO group's worth of compute.
Argument list too long: podman. This one is specific to the Podman backend: an oversized git patch generated by the agent can push the command-line arguments passed to the container runtime past the roughly 1MBARG_MAXlimit the OS enforces, and the rollout fails with an OS-level argument error rather than a training error.
None of these are theoretical edge cases if you're running the full launch config: at a train batch size of 16, a GRPO group size of 4, and 20 epochs, you'll generate thousands of rollouts over the course of a run, which is enough trajectories that all three failure modes will surface eventually. Catching them in the first few hundred steps, rather than after a multi-hour run completes, is the difference between a bad config costing you twenty minutes or costing you a full node-hour.
Checkpoint frequency matters here for a reason that has nothing to do with SkyRL specifically. On an earlier 8x H100 SXM5 spot node we ran for an unrelated pretraining job, the node had no storage volume attached at launch, and when it was reclaimed at step 956 the run had nothing to resume from: the cost that node had already run up bought nothing recoverable (as of 10-11 Nov 2026). SkyRL's default of checkpointing every 10 steps to ~/ckpts/llm_mini_swe is only useful if that path is on a volume that survives the node, spot or on-demand. Attach persistent storage before you launch, not after the first preemption.
Run Time and Cost on Spheron H100 Nodes
SkyRL doesn't publish a single canonical wall-clock number for the mini_swe_agent example, because run time depends on how many epochs you need before the eval reward curve plateaus, and on SWE-Gym's variable per-task runtime: a test suite that takes 90 seconds versus one that takes 8 minutes changes your rollout throughput directly. The published research does give a sense of scale at the H100-hour level: the SkyRL-Agent paper's SA-SWE-32B model, trained on a larger 2-node, 16-GPU setup against a different dataset (R2E-Gym, not SWE-Gym), used 4,601 H100 hours to reach 39.4% Pass@1 on SWE-Bench Verified, against 9,180 H100 hours for a comparable prior model, DeepSWE, at similar performance. That's a 32B model on a harder, larger dataset than the 8B/30B SWE-Gym configs this post covers, so treat it as a scale reference, not a cost estimate for your own run.
For a concrete anchor on what an 8x H100 SXM5 node costs, Spheron's H100 nodes support up to 8x H100 per node over NVLink/NVSwitch with 400 Gb/s InfiniBand and GPUDirect RDMA, tested with PyTorch DDP, DeepSpeed ZeRO, and Megatron-LM, which covers the FSDP-colocated shape SkyRL's launch scripts use. An 8x H100 SXM5 spot node runs $17.60/hr, or $2.20/hr per GPU; that's one snapshot, not a guaranteed figure. Rates move fast and don't move together across regions, so check the live rate for your region at deploy time rather than assuming last week's number still holds.
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 16 Nov 2026; other figures cited in this section are dated historical data points, not current rates, and may have changed. Check current GPU pricing for live rates.
Once a checkpoint clears your own eval bar, the next step is usually benchmarking it against SWE-bench and GAIA on a held-out split, rather than trusting the training-time eval numbers alone. And sizing the serving node is a different problem than sizing the training node: a 30B model trained across 16 H100s for RL can often be served at inference time on far less hardware, the way a 30B agentic coding model fits a single H100 once deployed rather than the multi-node footprint training needed.
Handing the Run to an Agent: SkyRL Training via the Agent GPU Sandbox
Everything above is a command you can run yourself from a terminal on a rented node. It's also a command you can hand to an agent instead. Spheron's Agent GPU Sandbox lets an MCP-connected client, Claude, Cursor, or Codex CLI among them, connect to connect.spheron.ai/mcp and provision its own GPU VM, then run commands with sudo and execute background jobs for up to 24 hours per command, streaming logs back into the chat session. Packages, models, and checkpoints stay on the VM between tasks and your own SSH key keeps working throughout, so you can tail the actual training log yourself instead of trusting only the agent's summary; full setup detail is in the sandbox docs.
A realistic prompt for the GSM8K sanity check looks like: "Deploy a 4-GPU node, clone NovaSky-AI/SkyRL, install it per the docs, and run the GSM8K GRPO quickstart. Report the reward curve every 20 steps." For the full mini_swe_agent run on an 8-GPU node, the 24-hour-per-command ceiling is the real constraint: a 20-epoch SWE-Gym job that runs longer than a day needs a shorter initial run to validate the config, or a follow-up command that resumes from the last checkpoint rather than one unbroken day-plus job. That same mechanics (background jobs, persistent checkpoints, streamed metrics) is what our walkthrough of training a reward model from one chat prompt uses for a smaller RL-adjacent job, applied here to a heavier, multi-GPU run. The sandbox's own feature page names reinforcement learning with SkyRL specifically among its documented use cases, alongside GRPO and RLVR training with verl, TRL, and OpenRLHF, so a SWE-Gym run like this one isn't an improvised stretch of the mechanics, it's what the product lists. The part still worth confirming yourself is capacity: check that your multi-GPU node is actually available in the dashboard before an agent kicks off a long run.
Every deploy, stop, start, and terminate still requires your explicit approval, and a connection can carry a spend limit per hour, so a runaway 20-epoch job can't silently eat a weekend's budget without you seeing the prompt first.
SkyRL turns training a coding agent into a GPU count and a launch script; Spheron's H100 nodes and Agent GPU Sandbox turn that script into a run you can launch, checkpoint, and hand to an agent without babysitting a terminal.
Frequently Asked Questions
SkyRL is a modular, full-stack reinforcement learning library from NovaSky-AI and Berkeley's Sky Computing Lab, used to train LLMs on long-horizon, multi-turn tasks: tool-using agents, coding agents that edit real repositories, and other sequential problems where the reward only arrives after a full trajectory of steps, not after one generation. Its codebase splits into skyrl-train for the training loop, skyrl-gym for Gymnasium-API tool-use environments, skyrl-agent for the long-horizon agent layer, and skyrl-tx for a Tinker-compatible backend.
It depends on model size. SkyRL's own mini_swe_agent example needs one node of 8x H100 GPUs to train Qwen3-8B on SWE-Gym, and two nodes of 8x H100 (16 GPUs total) for Qwen3-Coder-30B, per the launch scripts in the NovaSky-AI repo. A smaller sanity-check run, the GSM8K GRPO quickstart on Qwen2.5-1.5B-Instruct, needs only 4 GPUs and is worth running first to confirm your environment works before committing a full node to SWE-Gym.
SkyRL's published mini_swe_agent example trains on SWE-Gym, which supplies 2.4K real software engineering tasks drawn from 11 Python repositories plus a smaller 234-instance Lite split. The SkyRL-Agent research paper's SA-SWE-32B model instead trained on 4.5K instances from a different dataset, R2E-Gym, so which one a given SkyRL run uses depends on the example or recipe you start from, not on SkyRL itself.
Through verifiable, executable rewards rather than an LLM judge. The mini_swe_agent example runs each rollout's generated patch inside an isolated container, Podman by default, against the task's real test suite, and GRPO scores the rollout by whether the tests pass. SWE-Gym's own documentation also warns its environments contain a git shortcut exploit similar to a known SWE-bench issue, and recommends running its cleanup script before training so the agent can't reward-hack a passing grade without actually fixing the code.
Not for the SWE-Gym coding-agent example as published. The smallest documented coding-agent config, Qwen3-8B, needs a full 8-GPU H100 node, because SkyRL runs a trainer and 4 separate vLLM inference engines side by side even at that model size. SkyRL's GSM8K quickstart is the smallest documented multi-GPU entry point, at 4 GPUs, and is the better first run to verify your install before scaling up.







