DeepSeek-V4.1-Flash on one DGX Spark: making 510 GB work

A normal dense model uses every feed-forward block for every token. This model instead has 384 small specialist blocks—“experts”—in each of 40 layers and routes each token through only six. That leaves room for an unusual memory hierarchy.

Memory bars: green is the always-resident model core, blue or orange is the expert arena, and gray is working headroom.

The full-FP4 path is the faithful one: no expert is removed, and every expert keeps its native FP4 representation. An 81 GB arena holds 3,910 hot experts plus a 400-slot transient ring. A miss triggers a direct read from local NVMe. The transient ring is important during prompt processing, when many experts appear briefly; those one-off reads do not displace the hot decode set.

The faster path starts with a routing trace over 10,760 tokens. It keeps 154 of the 384 experts in every layer, then repacks those 6,160 experts into a per-row 3-bit codebook format called CB3. They occupy about 89 GB and stay resident. That removes expert SSD reads, but it also changes the model: 60% of the experts can no longer be selected, and the remaining expert weights are more compressed.

Short code and prose prompts generated 512 tokens; the random test used 8,172 prompt tokens and generated 256. Code and prose are medians of two runs. Random 8K is one run.

“Expert NVMe” is the engine's logical expert-read counter for that request, not total process I/O. The matched full-FP4 quality run separately recorded 3.818 TB of physical process reads.

Hover a bar for the exact measured value. Higher is better for decode throughput; lower is better for first-token time and storage traffic.

Both arms scored the same held-out tokens with the same dense FP4 components, FP8 output head, top-six routing, 512-token limit, and attention replay. Lower negative log-likelihood is better; higher top-one accuracy is better.

The penalty is modest on this corpus, but it is not zero—and it is larger on general text than code. These token-level measurements are a controlled comparison, not a substitute for full task benchmarks.

The trace ranks experts by how often the model used them. The blue line estimates a token-level LRU hit rate at each FP4 expert budget; orange asks a harder question: did all six experts needed by a layer hit together? The green line is the static share of routed calls covered by the ranked set.

The tooltip converts each expert count to its native-FP4 resident size. Around 4,000 FP4 experts use 75.2 GB and cover 74.4% of routed calls statically; the simulated token LRU hit is 84.8%. CB3 is not plotted on this FP4 memory curve because each kept expert is smaller and the other experts are deliberately unavailable.

Every benchmark phase had a five-second device monitor. Choose a phase below; temperature, power, and available-memory charts share a synchronized crosshair. Gaps in the source monitor remain gaps on the chart.

No phase recorded thermal throttling. “SW Power Cap” is the NVIDIA software power-limit flag, not a thermal limit.

The model checkpoint was 510,313,423,455 bytes and all 48 indexed weight shards were validated before the run. The software stack used PyTorch 2.13.0+cu130, Transformers 5.17.0, Triton 3.7.1, and the custom engine at commit 8bb38e1.

• The model can run on one Spark, but storage is part of the computer. Full FP4 generation is practical only because the engine treats unified memory and NVMe as one managed hierarchy.
• Expert locality is real enough to exploit. A 28% FP4 resident arena produced 82–89% measured expert hit rates on the three serving workloads.
• All-resident CB3 changes the bottleneck. It eliminated expert reads and lifted decode to 13.28–26.88 tok/s, but used nearly all available memory.
• The speedup has a quality price. The matched held-out test lost 2.01 top-one points on code and 2.64 on general text. Whether that is acceptable depends on the workload.
• This is a specialized research runtime. The result is interesting because it stretches one small system unusually far, not because it replaces mature general-purpose serving stacks.

One DGX Spark, one custom work-in-progress engine, two runs for each short workload, and one random-8K run per arm. The prompts are small diagnostic slices, not standardized downstream evaluations. CB3 changes both expert availability and expert precision, so its speed and quality effects cannot be assigned to either change alone. The routing trace and held-out quality corpus are finite samples; other domains may choose different experts and pay a different penalty.

Sources: DeepSeek-V4.1-Flash model card · the experimental DGX Spark engine. All charts and tables on this page use the measurements from the benchmark run described above.