The Model That Changed My Mind
I've been running local models for a while now. GLM-4.7-Flash, Minimax M2.5, Qwen3.5-122B — all solid, all useful. But none of them made me reconsider my hardware roadmap. DeepSeek-V4-Flash did.
I tested it first on my Frankenstein rig — the RTX PRO 6000 Blackwell (96GB) paired with the RTX 3090 that runs my desktop. I was running the Unsloth UD-IQ3-XXS quant, which is a severe 3-bit compression on a 284B-parameter MoE model. I was only able to get up to about 256k context, which reduced prefill speeds because I had to use aggressive batch settings, and generally a heavy quant at longer context kills quality. Compaction and context rot become more prone.
But even through that low-quant haze, the model's intelligence and tool use was unmistakable. It felt like Claude Sonnet 4.6. That's not a comparison I make lightly. There's a reasoning depth and instruction-following capability in this model that I simply haven't seen in models I've served locally, myself.
I had to know if it was the quant holding it back or the model itself. So I tested the full FP8 checkpoint in Opencode Go. Night and day. Fast, extremely cheap — I spent $0.19 for a morning pounding on the model, compared to about $5 for a similar session with Kimi-K3 — and delivering a quality I genuinely haven't seen in these middleweight models yet. The only thing missing was sovereignty over the weights.
The Numbers That Matter
The official FP8 checkpoint sits at roughly 162GB (284B parameters at 4-6 bits per parameter — it's approximately FP4 in size, not FP8 in the traditional sense). That leaves 96GB of VRAM in the cold — no amount of offloading or quantization gymnastics gets you there without significant quality loss. The math is simple: to run this model at native quality, you need more than 96GB of memory. Maybe a lot more.
I became convinced: the model is real. The bottleneck is hardware, not the weights.
The Decision
So I'm flipping the RTX PRO 6000. It's a phenomenal card — 96GB of GDDR7, built like a tank. But it's also a tank at ~600W, noisy, and with exactly 96GB of VRAM, which is just short of what I need.
I'm replacing it with two NVIDIA DGX Sparks. Each one has 128GB of unified LPDDR5x memory, 140W TDP, fanless/silent, and they pair over CX-7 200Gbps QSFP to form a single vLLM Tensor Parallel cluster. Total: 256GB combined memory, ~280W total power, silence, and the ability to run not just DeepSeek-V4-Flash at lossless FP8, but also whatever 120B+ models come next.
The resale on the 6000 (~$8-10K) covers the pair almost dollar-for-dollar. Net cost: effectively zero. To put that in perspective: a second RTX PRO 6000 would cost the same ~$8K for 192GB — but that's a noisy, 1200W hot rod with no published V4-Flash benchmarks. The Spark pair is a proven appliance with verified numbers: 41-45 tok/s decode, ~1750-1970 tok/s prefill, 1M context, 280W silent.
The Tradeoff
Let me be clear: two DGX Sparks will be slower than a single RTX 6000 on small models. The 6000 has 4x the compute and 6.6x the bandwidth of a single Spark. For models that fit in 96GB, the 6000 wins hands-down.
But the class of models I want to run no longer fits in 96GB. While the 27B to 120B class is very good for their size, they really can't hold their own with models in the ~300B range. And 128GB LPDDR5x unified, while slower than GDDR7, buys me the capacity to load the lossless FP8 checkpoint with room for KV cache at 1M context. Capacity is the constraint, not speed. More capacity means access to a higher class of model.
Small models will stay on my 3090 (desktop + embedding server). The Sparks will handle my deep work. And if I need to scale further, the 2-node pair can expand to 4 directly (never 3 — 256-expert divisibility is real) with a 200G switch.
Why This Matters
DeepSeek-V4-Flash at FP8 quality, running on my own hardware, under my own control, with 1M context, at 280W silent — that's the vision. DeepSeek V4 Flash is the best local model I've ever used, and the hardware path to owning it is now clear. Two DGX Sparks give me sovereignty over weights that rival closed-source API quality, at a cost that's effectively a wash after selling the 6000.
I've been building toward this for months. The Ironhorse rig taught me multi-GPU inference, and I got experience tuning some of the best small models like Qwen and Gemma. This is the next chapter: from an open-air hydra to mini AI lab for the evermore capable models.
Let's see what 256GB of unified memory gets you.
—Ash