DGX Spark 4-Node Cluster: 512GB for 700B Parameter Models (2026)

The 2-unit DGX Spark story stops at 128GB to 256GB, and most of what's written about clustering these boxes stops there too. NVIDIA's Sync Cluster Assistant goes further: it officially supports pooling up to four 128GB units into 512GB of unified memory, enough to put a 500B-to-700B-class MoE model on a desk instead of in a data center. Almost nobody has worked through what that fourth node actually requires, what it measurably runs, or the dollar point where it stops being cheaper than just renting the GPU.

TL;DR: DGX Spark 4-Node Cluster for 700B-Class Models

A 4-node DGX Spark cluster pools four 128GB units into 512GB of unified memory using NVIDIA's Sync Cluster Assistant, officially over a QSFP switch rather than a switchless ring, and has been measured running DeepSeek V4.1-Flash (552B total, 476GB footprint) at up to 494 tokens/second.

• Capacity: four 128GB DGX Spark units pool to 512GB of unified memory, 16TB of combined NVMe storage, and 4 petaflops of combined FP4 compute.
• Topology: NVIDIA's officially supported path for 4 nodes is a QSFP switch, since each GB10 chip's two ConnectX-7 ports can't form a working ring at 4 nodes the way they can at 3.
• Measured: DeepSeek V4.1-Flash hit a peak of 494 tokens/second at 32 concurrent requests on a 4-node switchless-ring build, documented by hardware analyst Patrick Moorhead.
• Price: a 4-node, 512GB build runs roughly $25,000-$40,000 in hardware, before counting the switch.
• Cloud alternative: Spheron's B200 on-demand instances run $10.05/hr ($5.22/hr spot) as of 21 Oct 2026, billed by the minute, for workloads that outgrow the cluster's ceiling or only need to run occasionally. Compare live GPU pricing.

DGX Spark Clustering Guide: From a Two-Unit Sync Pair to a Four-Unit Fabric

NVIDIA's Sync Cluster Assistant is the software that turns two to four separate DGX Spark boxes into one addressable memory pool for large models and multi-agent pipelines, according to NVIDIA's own developer blog. The two-unit version is well covered: our DGX Spark 64GB Sync Clustering guide walks through pairing two units over one QSFP cable to reach 128GB and 546 GB/s of combined bandwidth, a direct connection with no switch and no routing complexity. Three physical topologies are supported overall: a two-node direct connection over a single cable, a three-node ring using three QSFP cables with both ConnectX-7 ports active on every node, and a two-to-four node fabric built through a QSFP switch, per the same NVIDIA developer blog post.

The four-node tier is where the hardware stops being a simple extension of the two-node trick, and where most writing on DGX Spark clustering quietly stops.

Why a 4-Node Switchless Ring Isn't the Same Trick as a 3-Node Ring

A 3-node ring works without a switch because the geometry is complete: three nodes, three cables, every node directly connected to both of its neighbors. A 4-node ring looks like it should work the same way, but a hardware limit that doesn't show up until the fourth unit joins gets in the way: each GB10 chip has exactly two ConnectX-7 ports, as explained in a detailed topology writeup from Petronella Technology Group. In a ring of four nodes (Spark 1 through 4, connected 1-2-3-4-1), every node is only directly cabled to its two immediate neighbors. Node 1 and Node 3 sit across the ring with no cable between them, and the same is true for Node 2 and Node 4.

That's a routing problem, not a bandwidth one. Traffic between two non-adjacent nodes has no direct layer-2 path, so RDMA between them fails outright unless you bring in a patched NCCL build and the extra routing logic yourself, which is the unsupported workaround Petronella's writeup flags as fragile. A 3-node ring has no such gap, because every node in a triangle is adjacent to every other node by definition. Four is the first node count in this family where "just add another cable" stops being true, which is the detail most 2-node and 3-node clustering comparisons skip past.

The Switch-Based Path NVIDIA Actually Recommends

NVIDIA's documented route for connecting two to four nodes is a QSFP switch, with at least four QSFP56-DD ports running at 200Gbps each, according to Exxact's build guide for a 4x DGX Spark cluster. A switch turns the routing gap above into a non-issue: every node connects to the switch instead of to each other, so there's no non-adjacent pair and no missing path.

Exxact's guide names a concrete part: a MikroTik CRS804-4DDQ-HRM, which gives four 400G QSFP56-DD ports plus two 10GbE management ports, stays under 150W of TDP, and fits in a single rack unit, a small, cheap addition relative to the four DGX Spark units it connects. Power is the other number worth checking before committing to this build: four DGX Spark units draw roughly 960W total (about 240W each), and the switch adds another 92-123W, keeping a full 4-node cluster under 1,200W, per the same Exxact build guide, comfortably inside a standard household or office circuit. For readers planning to run either topology at sustained, multi-hour load, our DGX Spark thermal throttling and power draw measurements cover what clock decay looks like on a single unit before you multiply that risk by four.

Once four nodes are cabled and talking, the fabric itself is worth verifying rather than trusting on faith. The same all_reduce_perf acceptance test used on rack-scale InfiniBand clusters applies at this smaller scale too; our NCCL tests guide for benchmarking cluster interconnect covers building and reading that benchmark, and it's the only way to confirm your 4-node fabric is actually moving data at the rate the switch or ring topology promises on paper.

512GB Unified Memory: What Actually Fits at FP4 and NVFP4

A single 128GB DGX Spark already tops out somewhere around 100B parameters at full precision, covered in our 128GB fine-tuning memory breakdown. A 4-node, 512GB pool moves that ceiling considerably further, but the honest answer to "what fits" depends on whether the model is dense or MoE, because sparsity is what actually makes a 500B+ model workable on this hardware.

Pooled four-node capacity breaks down as 512GB of unified memory, 16TB of combined NVMe storage (4TB per node), and 4 petaflops of combined FP4 tensor throughput. Here's what that pool measurably and plausibly holds:

DeepSeek V4.1-Flash is the one entry with a real, measured number behind it: the model used a 476GB memory footprint within the roughly 512GB pool, on a 4-node switchless-ring cluster. That's roughly 36GB of margin, thin for a production deployment but workable for the single-stream or low-concurrency serving a desk-side cluster is built for.

DeepSeek-R1, at 671B parameters, is the model most commonly named as the realistic target for this tier at 4-bit or NVFP4 quantization, sitting inside the 700-800B ceiling the combined memory and compute can address. No one publishing on this hardware has put a specific measured footprint next to R1 the way wccftech's coverage did for V4.1-Flash, so treat that row as the documented upper bound this hardware is expected to hold, not a confirmed measurement.

The pattern across both rows: it's MoE sparsity, not raw parameter count, that makes a 500-700B model fit a 512GB pool at all. A dense model of the same size at the same 4-bit precision would need a similar byte count, but without MoE's lower active-parameter compute cost, decode throughput on four GB10 chips would be far slower than the next section measures.

Measured: DeepSeek V4.1-Flash Throughput at TP=3 vs TP=4

Capacity answers whether a model fits. Throughput answers whether it's usable once it's loaded, and this is where the fourth node earns its keep in a way a spec sheet alone doesn't show.

The MiaAI-Lab project benchmarked DeepSeek-V4.1-Flash across both 3-node and 4-node DGX Spark configurations, using MXFP4 expert weights and NVMe-resident Engram tables to fit the model, according to the project's own published results. The gap between TP=3 and TP=4 is not a marginal one:

Going from three nodes to four nearly doubles single-stream decode, from 51.0 to 87.7 tokens/second, and more than doubles the 4-concurrent-stream aggregate, from 75.1 to 163.6 tokens/second. That's a better-than-linear jump for one-third more hardware, which tracks with the model's MoE structure: a fourth node adds memory bandwidth and a better expert-routing balance, since the 552B-parameter model's experts spread more evenly across four pools instead of three.

For context, a single DGX Spark running Llama 3.3 70B decodes at roughly 4.7 tokens per second, measured and published by willitrunai.com. That's a smaller, dense model on one unit, not an apples-to-apples comparison with a 552B MoE model on four, but it's the baseline this post's cluster math scales up from: four networked GB10 chips, running a model roughly 8x larger, reach double and eventually triple digits of tokens per second as concurrency rises, something a single unit at this model class could never approach.

The 494-tokens-per-second peak figure cited earlier sits above MiaAI-Lab's 16-stream aggregate of 342.7 tokens/second, consistent with a different concurrency point (32 concurrent requests) and serving configuration rather than a contradiction. Both point the same direction: a 4-node cluster scales throughput on this model well past three nodes or one, because DeepSeek V4.1-Flash's MoE design lets each additional node absorb a proportional share of active experts rather than just adding idle memory.

Where a 4-Node Cluster Stops Making Sense vs Renting a Cloud GPU

A documented 4-node, 512GB switchless-ring build runs roughly $25,000 to $40,000 in hardware, per Patrick Moorhead's build writeup, before adding a switch for the officially supported path. That's a serious outlay for hardware that caps out around 700-800B parameters and, per the table above, runs with single-digit-percent memory headroom on at least one documented model. The question worth asking isn't whether it works, the benchmarks above confirm it does, it's whether the workload needs to live on a desk at all.

Our DGX Spark break-even hours analysis runs this math for a single unit against H100, H200, and B200 on-demand rates, and the same logic scales up here: a $25,000-$40,000 cluster only beats renting if it runs enough hours, at high enough utilization, that owning it drops below the cloud alternative's cost. Four idle units on a desk most of the week are a worse deal than a handful of billed cloud hours; four units running near-continuously for a workload that genuinely needs offline, private, no-meter-running access are a different calculation.

What a cluster this size cannot do is compete on concurrency. Running DeepSeek V4.1 Flash on 8xB200 on Spheron as one 4-GPU copy with FP8 and FP4 precision, 1M context, image input, and speculative decoding, measured warm on 15 Oct 2026, a single user gets 500 tokens/second, dropping to 260 at 8 concurrent users, 202 at 16, and 170 at 32, with a 0.45-second time to first token. Under a 32-session agent load on that same cloud copy, the node read 218,578 prompt tokens/second and wrote 2,555 output tokens/second at 86% cache hit rate, with zero errors. That's the same model family the DGX Spark cluster is running, at a concurrency no desk-side cluster reaches: the cloud node absorbs 32 simultaneous sessions with graceful degradation, where a 4-node cluster running at single-digit memory margin is built around one or a handful of users at a time.

Spot capacity deserves a specific caveat rather than a blanket recommendation. During inference testing, every 8-GPU spot type in use was reclaimed at least once, and in one region five spot VMs were stopped in a single day. That reliability profile fits short, fault-tolerant bursts, not the long-running, unattended job a 4-node DGX Spark cluster is actually good at. Choosing cloud spot as the alternative to buying a desk-side cluster means keeping a second copy of the model on a different node or region with automatic recovery built in.

Pricing fluctuates based on GPU availability. Spheron rates above are live as of 21 Oct 2026; other figures in this post reflect their sources' most recently published data and may have changed since. Check current GPU pricing → for live rates.

Moving Past 512GB: The DGX Spark to Cloud B200/GB200 Path

A 4-node cluster's roughly 700-800B parameter ceiling is a real wall. Past it, whether the target model is a trillion-parameter MoE, the workload needs more than single-digit-user concurrency, or uptime can't depend on four desk units staying cool and powered, the next step isn't a fifth DGX Spark, it's cloud GPU infrastructure built for that scale.

For models that still fit a single large multi-GPU node, our DeepSeek V4-Pro deployment guide covers running a 1.6T-parameter MoE model from the same DeepSeek family on a single 8x H200 node, the setup a DGX Spark cluster hands off to once its own ceiling is reached. Spinning that up isn't instant the way flipping on a desk-side cluster is: cold starts on 8xB200 spot, from launch to serving with weights and compiled kernels loading onto a cache volume, measured 15-16 Oct 2026, ran 11.9 minutes for DeepSeek V4.1 Flash and 15.9 minutes for GLM-5.3, with getting the VM itself accounting for 2 to 3.5 minutes of that, a fraction of the lead time a new DGX Spark purchase and build carries, and a cost that applies once per deployment rather than per request.

For a broader framing of when a cluster beats a single large GPU, our multi-GPU vs single-GPU cluster guide covers the VRAM-first decision logic this post applies specifically to DGX Spark. For workloads that outgrow even a multi-GPU cloud node, trillion-parameter MoE models or dense frontier models needing rack-scale NVLink, our GB200 NVL72 guide covers the tier above everything here. GB200, GB300, and Vera Rubin capacity on Spheron is reservation inventory rather than an on-demand hourly rate, so a reader scaling toward rack-scale hardware should expect a reservation conversation through the GB200 page, not an instant checkout. For the local-to-cloud handoff itself, our DGX Spark to GPU cloud pipeline guide covers the Docker and vLLM steps, and Spheron's SSH connection docs cover getting into a provisioned instance once it's up.

None of this makes the 4-node cluster the wrong purchase for the right workload. A team that's validated a 500-700B MoE model fits comfortably, needs offline access, and doesn't need more than a handful of concurrent users has a real, working, once-paid-for setup on its desk. The job stops exactly where this section starts: once the model, concurrency, or uptime requirement outgrows four GB10 chips, no fifth unit fixes that, and the honest next step is a rented or reserved GPU built for the scale the desk cluster was never meant to reach.

A 4-node DGX Spark cluster tops out around 700-800B parameters and a handful of concurrent users. Past that ceiling, Spheron's on-demand B200 instances pick up the same workload the same day, billed by the minute, no hardware build required.

Check live B200 pricing on Spheron →