DGX Spark fine-tuning starts with a VRAM formula and gets complicated fast. Run the same LoRA and QLoRA math this site already uses for discrete GPUs against DGX Spark's real, OS-reported memory pool instead of the 128GB marketing number, and the ceiling moves. Then read the GB10 developer forum, where owners have logged a fine-tune hanging for nine hours at 5% GPU utilization, and a 17-hour run that runs out of memory three-quarters of the way through, and the ceiling moves again, in the opposite direction from what the spreadsheet says.
This piece takes the sizing formulas from our GPU VRAM requirements guide and re-runs them against DGX Spark's actual usable pool, then puts the forum-documented failures next to that math so you know which gap is real before you commit a multi-hour job to the box.
TL;DR: DGX Spark Fine-Tuning in 128GB
- Usable memory: DGX Spark's unified pool reports roughly 119.68GB to the OS, not the full 128GB advertised, a GB-vs-GiB accounting gap, not a hardware shortfall.
- QLoRA headroom: a 70B dense model QLoRA fine-tune needs roughly 52GB, about 67GB under the pool's ceiling.
- LoRA's wall: a 70B standard LoRA run needs roughly 159GB, more than DGX Spark holds at all; a 2x H100 SXM5 cloud node can.
- Formula vs reality: a gpt-oss-120B QLoRA job documented at 65GB failed to even load on DGX Spark, and a Qwen 2.5 14B run OOM'd after 17 hours at 70% complete.
- Spheron: rents H100 and H200 GPUs by the minute for fine-tunes that outgrow one DGX Spark. Check H100 GPU rental availability.
What DGX Spark's 128GB Unified Memory Actually Gives You
DGX Spark pairs an Arm-based Grace CPU with a Blackwell GPU on one GB10 superchip, and both sides draw from a single LPDDR5x pool instead of separate CPU RAM and GPU VRAM connected over PCIe. For fine-tuning, that matters in one specific way: there's no host-to-device copy step and no discrete VRAM wall at 80GB or 141GB the way there is on an H100 or H200. The whole pool is addressable from the training process without a transfer.
Three things complicate that headline advantage, though.
First, the usable number is smaller than the marketing number. DGX Spark's spec sheet advertises 128GB. The OS reports roughly 119.68GB actually available, a gap that comes from decimal gigabytes (what the spec sheet counts) versus binary gibibytes (what the kernel reports), not from a defect or reserved partition. Every VRAM budget in this piece uses the 119.68GB figure, not 128GB, because that's the number your training process actually gets to work with.
Second, that pool is shared with everything else on the box. A discrete GPU's VRAM is isolated: the OS, your browser, and your data-loading pipeline live in system RAM, completely separate from what CUDA sees. On DGX Spark, the CPU side of your own training job, the OS, and whatever else is running on the desk unit all draw from the same physical pool the GPU needs. A formula that assumes the full 119.68GB is available exclusively to the GPU context is already optimistic before a single batch runs.
Third, there's no discrete VRAM wall, but there's a bandwidth wall instead. DGX Spark's GB10 chip delivers roughly 273 GB/s of memory bandwidth, against an H100 SXM5's 3.35 TB/s, a gap that compounds the longer a fine-tuning job's sequence length or batch size grows. Generous capacity doesn't buy you out of a throughput ceiling; it just moves where you hit it. For raw tokens-per-second comparisons against a discrete GPU at the same price point, see our DGX Spark vs RTX 5090 benchmark. And DGX Spark isn't the only box built on this chip: every certified GB10 system from ASUS, Dell, HP, and Lenovo shares this identical 119.68GB ceiling and 273 GB/s bandwidth, so the math in this post applies regardless of which GB10 box is on your desk.
Full Fine-Tune vs LoRA vs QLoRA: What 128GB Actually Buys You
Run the standard fine-tuning VRAM formula against DGX Spark's 119.68GB usable pool and QLoRA covers everything up to a 70B dense model with real headroom left over, standard LoRA stops working past roughly 32B, and full fine-tuning never clears single-digit billions of parameters before the Adam optimizer state alone exceeds the whole pool.
| Model Size | Full Fine-Tune | LoRA (r=64, BF16) | QLoRA (4-bit NF4) | Fits in DGX Spark's 119.68GB Pool? |
|---|---|---|---|---|
| 7B/8B | ~88 GB | ~20 GB | ~8 GB | Full fine-tune, LoRA, and QLoRA all fit |
| 14B | ~174 GB | ~35 GB | ~14 GB | LoRA and QLoRA fit; full fine-tune does not |
| 32B | ~394 GB | ~76 GB | ~28 GB | LoRA and QLoRA fit; full fine-tune does not |
| 70B/72B | ~860 GB | ~159 GB | ~52 GB | Only QLoRA fits |
| MoE 30B A3B | ~105 GB | ~69 GB | ~21 GB | LoRA and QLoRA fit; full fine-tune leaves only ~15GB spare |
These figures carry over directly from the full VRAM breakdown by model and method: weights plus gradients plus Adam's two FP32 moment buffers plus activations at batch size 1, sequence length 512. The only thing that changes on DGX Spark is the ceiling you're checking against, 119.68GB instead of a GPU's labeled VRAM.
Two rows are worth a second look. The 70B/72B LoRA row needs roughly 159GB, which is larger than DGX Spark's entire pool, not just tight against it. There's no gradient checkpointing trick that closes a 40GB gap when checkpointing only touches activation memory, not the base weights, adapter gradients, and optimizer states that make up the non-activation floor. And the MoE 30B A3B full fine-tune row technically fits at ~105GB, but leaves only about 15GB for the OS, data loading, and any background process competing for the same physical pool, which is a thinner margin than the number alone suggests. If standard LoRA's memory floor is the problem, adapter variants like DoRA and PiSSA can shift that floor; see our PEFT methods guide for where each one helps.
Measured: The Largest Model You Can LoRA/QLoRA Fine-Tune Without Offloading
The formula says QLoRA scales comfortably to 70B dense models and, on paper, to much larger sparse MoE models too, since only active parameters need gradients and optimizer state. The forum record backs part of that up and contradicts the other part sharply.
On the confirmed-working side, Unsloth's gpt-oss-20B QLoRA fine-tuning fits in about 14GB of memory, well inside DGX Spark's pool with more than 100GB to spare. That number lines up with where the formula puts a 14B-class QLoRA job, and multiple owners have run it without incident using Unsloth's GB10-specific playbook.
The failure modes documented on the GB10 forum, however, point at how the model loads: before 4-bit quantization finishes compressing the weights, the loader needs a larger, cleaner allocation than a static formula predicts, and on a shared unified-memory pool without a dedicated VRAM BAR, that allocation step is a different, harder problem than it is on a discrete GPU.
The practical answer to "what's the largest model you can actually fine-tune on DGX Spark without offloading" is narrower than the 119.68GB number suggests: dense models up through 70B with QLoRA are confirmed to work by both the formula and real reports, and smaller MoE models in the gpt-oss-20B class work too. Above that, formula-fits is necessary but not sufficient; treat any specific larger model as unverified on your own unit until you've actually run it. Unsloth and TorchTune are the two frameworks behind most of these GB10 reports; see our Axolotl vs Unsloth vs TorchTune comparison if you're choosing between them for your own run.
Where the Math Lies: Jobs That "Fit" on Paper but Fail on DGX Spark
The gpt-oss-120B load failure above is the cleanest example of formula-fits-but-reality-doesn't, but it isn't the only one, and the other two failure modes aren't even about VRAM capacity at all.
A separate gpt-oss-20B fine-tuning run on DGX Spark hung at step 165 of 1,000 after 9 hours, with CPU usage pegged at 100% and GPU utilization stuck at 5%. This is the same model class that runs fine at 14GB elsewhere, so it isn't a capacity problem. It's a contention problem: on a unified-memory system, the CPU-side work in a training loop, data loading, tokenization, preprocessing, all compete for the same physical memory bus the GPU needs to pull weights and activations from. A VRAM formula only accounts for what the GPU needs to hold; it says nothing about what else is fighting for bandwidth on the same pool, and no formula in this piece or the sister VRAM guide captures that.
In the same forum thread, a different user's Qwen 2.5 14B fine-tune ran successfully for 17 hours and then failed with an out-of-memory error at 70% completion. QLoRA on a 14B model needs roughly 14GB by the formula, against a 119.68GB pool, over 100GB of apparent headroom. A run that OOMs three-quarters of the way through a 17-hour job, with that much margin on paper, is the clearest evidence that the batch-size-1, sequence-length-512 formula snapshot is a floor at the start of training, not a guarantee across its full length. Checkpoint saves, periodic evaluation passes, and memory fragmentation accumulate over a multi-hour run in ways a single static calculation never captures, and on a shared pool, there's no separate system-RAM overflow to catch the surplus the way there might be on a workstation with a discrete GPU and abundant host RAM alongside it.
The common thread across all three cases: the VRAM formula is necessary for sizing a job correctly, but it was never designed to predict loader behavior, CPU/GPU memory contention, or multi-hour drift, and DGX Spark's shared-pool architecture makes all three bite harder than they would on a discrete GPU with memory that's walled off from the rest of the system.
Where It Breaks: Long Sequence Length, Multi-Epoch Runs, and the Thermal Ceiling
Two more failure surfaces sit outside the VRAM formula entirely, and both get worse the longer a job runs.
Long sequence length hits the bandwidth ceiling before it hits the memory ceiling. KV cache and activation memory scale with sequence length, quadratically without flash attention, linearly with it, and that's true on any GPU. What's different on DGX Spark is that 273 GB/s of memory bandwidth is a fraction of an H100 SXM5's 3.35 TB/s, so a longer context window starts bottlenecking throughput well before it exhausts the 119.68GB pool. A job that "fits" by the VRAM table can still slow to a crawl at long context lengths in a way the same model wouldn't on a bandwidth-rich discrete GPU.
Multi-epoch, multi-hour runs run into DGX Spark's thermal and power ceiling, a separate failure mode from anything in the VRAM math. We measured this directly: sustained inference and training loads on GB10 hardware show real clock decay and, on some units, hard power-offs well before a job completes. The fix we found, locking GPU clocks with nvidia-smi -lgc, trades a modest throughput hit for a job that actually finishes instead of crashing partway through. If you're planning a fine-tuning run that spans multiple hours or multiple epochs unattended, read our DGX Spark thermal throttling and power draw measurements before you kick it off. A job that's correctly sized on VRAM can still fail on heat.
The Local-to-Cloud Handoff: When to Burst a DGX Spark Fine-Tune to a Rented GPU
The honest line between "run it on DGX Spark" and "rent a GPU for this one" comes down to three questions: does the method and model size actually fit the 119.68GB pool with real margin, is the job short enough to finish in one sitting, and can you tolerate the unverified-until-tested risk documented above.
Keep it local when: you're running QLoRA on anything up to a 70B dense model (~52GB, 67GB of headroom) or a confirmed-working MoE class like gpt-oss-20B (~14GB), the job completes in a few hours rather than running unattended overnight, and you've already validated that specific model and framework combination loads and trains cleanly on your unit. Raschka's own framing is worth repeating here: "I see it more as a development and prototyping system... I consider it as an in-between machine that I can use for smaller runs." For the same purchase decision from the other competing desk-side box, see our DGX Spark vs Mac Studio comparison.
Burst to a rented GPU when: the method needs standard LoRA or a full fine-tune past 7B (both exceed the 119.68GB pool at 14B and up), you're training a model class in the 120B+ range where DGX Spark's own forum shows real load failures even when the formula says it should fit, or the run is long enough that a hang, an OOM, or a thermal shutdown would waste more compute time than renting outright. As Raschka puts it plainly: "It's not a replacement for A100 or H100 GPUs when it comes to large-scale training."
For the LoRA case specifically, a 70B run that needs roughly 159GB fits on 2x H100 SXM5 ($5.50/hr, $2.75/hr per GPU), which has enough combined VRAM to hold the sharded weights, adapter gradients, and optimizer states that a single DGX Spark cannot. For a full fine-tune at that scale, the VRAM guide's math works out to 11x H100 SXM5 with FSDP ($30.25/hr). For single-GPU jobs that need more headroom than one H100's 80GB without going multi-GPU, an H200 SXM5's 141GB (from $3.32/hr) is the next step up. Spot capacity suits the long, fault-tolerant runs this piece shows DGX Spark struggling to finish unattended, provided you enable checkpointing before you start and accept that the capacity can be reclaimed without notice: H100 spot runs $2.21/hr against $2.75/hr on-demand, and the two track different supply pools, so check each tier's live rate rather than assuming one is cheaper.
Two honest limits worth stating plainly. Spheron doesn't sell or rent the DGX Spark or any GB10 unit itself, so a decision made purely on local-hardware economics, the one-time purchase price against years of dev usage, needs to be weighed against hourly cloud rates on its own, not shopped for here. And for jobs that already fit comfortably inside DGX Spark's pool and finish in one sitting, renting a cloud GPU is a net cost increase with no local, offline benefit, not an upgrade. For the full local-to-cloud workflow and a worked break-even calculation on dev-hour cost, see our DGX Spark and GPU cloud pipeline guide, which this post completes on the memory-sizing side.
Pricing fluctuates based on GPU availability. Spheron rates above are live as of 13 Oct 2026; other figures in this piece reflect their sources' most recently published data and may have changed since. Check current GPU pricing → for live rates.
A fine-tune that formula-fits on DGX Spark's 119.68GB pool is still worth testing before you trust it with a multi-hour job. When it doesn't fit, or the run is too long to risk locally, Spheron's H100 and H200 instances give you the multi-GPU headroom DGX Spark can't, billed by the minute.
Frequently Asked Questions
It depends on the method, not just the model size. QLoRA needs roughly 8GB for a 7B model up to about 52GB for a 70B dense model, both well inside DGX Spark's ~119.68GB usable pool (out of the 128GB advertised). Standard LoRA holds up through about 32B (~76GB) before it runs out of room; a 70B LoRA run needs roughly 159GB, which exceeds what a single DGX Spark has even before anything else on the box competes for the same shared memory.
Yes with QLoRA, no with standard LoRA or a full fine-tune. A 70B QLoRA run needs roughly 52GB, comfortably under DGX Spark's 119.68GB pool with about 67GB to spare. Standard LoRA needs roughly 159GB and a full fine-tune needs roughly 860GB across 11 H100-class GPUs with FSDP, neither of which fits on one unified-memory box. For the LoRA case, 2x H100 SXM5 on Spheron ($5.50/hr, $2.75/hr per GPU) is the smallest cloud configuration that holds it.
Both, within real limits. [Sebastian Raschka's pretraining, supervised fine-tuning, and DPO benchmarks on a 355M-parameter model found DGX Spark and an A100 both significantly faster than a Mac Mini across all three stages](https://sebastianraschka.com/blog/2025/dgx-impressions.html). But Raschka, who ran those tests, frames the box itself plainly: "I see it more as a development and prototyping system... I consider it as an in-between machine that I can use for smaller runs," and adds: "It's not a replacement for A100 or H100 GPUs when it comes to large-scale training."
With standard LoRA (not QLoRA), the formula caps out around 32B parameters at roughly 76GB, leaving real headroom inside the 119.68GB pool. A 70B LoRA run needs roughly 159GB, past what one DGX Spark can hold regardless of offloading tricks. QLoRA is the method that actually gets 70B-class dense models training locally, and [forum reports confirm smaller MoE models like gpt-oss-20B running comfortably inside 14GB with QLoRA](https://forums.developer.nvidia.com/t/unsloth-finetuning-playbook-fine-tuning-gpt-oss-20b-with-gb10-forum-data/352770).
