Hardware profile
Own the middle.
DGX Spark is not a faster graphics card. It is a different shape of machine: one pool of memory big enough for models a consumer GPU cannot hold at all.
Why unified memory matters
A consumer GPU keeps model weights in its own VRAM. When a model does not fit, it spills into system RAM over PCIe and slows to a crawl. A unified-memory system gives CPU and GPU one large pool — the model simply fits. Capacity, not raw speed, is what it buys you.
| Metric | DGX Spark | RTX 4090 |
|---|---|---|
| Memory | 128 GB | 24 GB |
| Bandwidth | 273 GB/s | 1008 GB/s |
Roughly five times the memory, roughly a quarter of the bandwidth. Spark runs models a discrete GPU cannot hold at all — but on a model both can hold, the GPU generates tokens faster.
NVIDIA's published claims
- Inference, one unit
- Models up to 200B parameters
- Fine-tuning, one unit
- Models up to 70B parameters
- Two units linked
- Models up to 405B parameters
Our own arithmetic
Not an NVIDIA statement: 200 billion parameters only fit in 128 GB at roughly 4-bit precision (200B × 0.5 bytes ≈ 100 GB of weights) before any context memory. NVIDIA does not publish the precision behind that number, so read “200B-class” as heavily quantized.
Honest caveats
- Capacity is not speed: memory bandwidth is well below a high-end discrete GPU, so tokens per second on a model that fits both will favour the GPU.
- It is an ARM64 machine. Most mainstream AI tooling supports it, but niche x86-only packages may need work.
- Vendor model-size claims assume aggressive quantization. A '200B-class' model means heavily quantized, not full precision.
- If your heavy jobs are occasional, renting GPU hours is almost always cheaper than buying.
Why we publish no tokens-per-second figure
Tokens per second depend on the model, its quantization, the context length and the runtime — and every runtime update moves the number. A single figure would be wrong for most readers and stale within weeks, so this site publishes none. It measures what it can measure on a schedule: memory, prices, sizes.
What can be said comes from physics, not benchmarks. Generating a token means reading most of the model's active weights out of memory, so memory bandwidth is the hard ceiling on speed. DGX Spark's 273 GB/s against roughly 1,000 GB/s on a top consumer graphics card means that on a model which fits both, the card's ceiling is about four times higher — real runtimes reach only a fraction of either ceiling, but the ratio between them holds.
The consequence: Spark buys capacity — models that would not fit a consumer card at all — not speed. If you want fast output on a model that already fits a GPU, the GPU wins; if you want a model that needs 100 GB of memory to exist on your desk at all, nothing consumer-priced competes.
Bandwidth figures are vendor specifications (NVIDIA: 273 GB/s for DGX Spark; 1,008 GB/s for the GeForce RTX 4090). The ratio is arithmetic on those two numbers, not a measurement of any model.
Two Sparks, or four?
NVIDIA's own product page states that its ConnectX networking lets up to four DGX Spark systems be connected to work with models of up to 700 billion parameters. That is a capacity claim, and it is real: four pooled machines hold four times the weights.
It is not a speed claim. The link between boxes is a network cable, orders of magnitude slower than the memory inside each box, so a model split across units generates at the pace of that link, not of the chips. And the parameter figure carries the same fine print as the single-box claim: it assumes heavy quantization, and the precision behind the number is not stated.
Honest reading: linking Sparks is for people who need a very large model to fit at all and can accept slow output. If the goal is speed on a model that fits one box, the second box buys nothing — and for an occasional big job, an hour of a rented data-center GPU is cheaper than a second machine.
Source: NVIDIA DGX Spark product page ("up to four NVIDIA DGX Spark systems … AI models of up to 700 billion parameters").
What fits in 128 GB
Context length: 8K tokens · f16 (default)
All memory figures are estimates: measured quantized file size + computed context memory + runtime overhead, with a 12% safety margin on your hardware. Real usage varies with runtime version and settings.
Spark or rent?
Buying makes sense when large local models are your daily routine and your data must stay in the building. If the heavy work happens a few times a month, renting is cheaper — the break-even maths is on the rental page.
Break-even →Who it is for
- + You run 70B-class or larger models locally, most days.
- + Your data cannot leave your desk.
- + Capacity matters more to you than tokens per second.
- + You are comfortable on an ARM64 Linux machine.