experimental / september 2026

cuda.fast on DGX Spark: real speedup, narrow claim

The cuda.fast leaderboard says Qwen3.8-Flash-Next now runs 108.5% faster on one DGX Spark. I built the exact promoted engine, verified the pinned 109 GB GGUF snapshot, ran its public harness three times, and recorded the entire box every five seconds.

The result is interesting for a reason bigger than one score: this is a purpose-built inference engine that keeps the useful weights resident, reads a giant lookup table from NVMe, and uses Qwen's own MTP head to finish more than one token per expensive model step.

official record

108.5% composite increase

my decode mean

32.84 tok/s

MTP acceptance

90.91%

SSD-resident table

26.82 GiB

correctness

3 / 3 passed

the verdict

The number is real inside the benchmark, but the sentence is easy to overread. The official result is paired against a serial control on the same Spark, in the same job, with two control/candidate pairs and a calibrated health band. That is stronger than comparing against an old number from another machine.

What it does not mean is twice the speed for every prompt, context length, concurrency level, or serving stack. It is a decode-heavy composite for one single-stream benchmark shape.

credible, narrowly scopedNot inflated; insufficiently qualified if the score formula is omitted.

what is actually different

This is not vLLM with another launch flag, and it is not a general OpenAI-compatible server. The measured path is a compact engine and benchmark adapter built around this model and this GB10.

1 · GGUF shardsUnsloth UD-Q4_K_XL, checked byte-for-byte before launch.

2 · split residency76.87 GiB stays resident; the 26.82 GiB PLE table is read directly from NVMe.

3 · ds4 on GB10Custom CUDA kernels for Qwen's GDN, sparse attention, routing and quantized experts.

4 · native MTP headA 2.60 GiB Q8 head proposes the next token while the target model verifies it.

5 · resident protocolOne model load serves every harness phase through a local socket.

The promoted commit adds a GDN kernel that reuses one query/key vector and gate pair across four value rows. That saves repeated memory work without changing the ordered arithmetic for each row. The larger win is cumulative: model-specific kernels, SSD-aware layout, one-load residency, and speculative decode all compound.

my three runs

The public harness uses a 1,024-token long-copy prompt, greedy single-stream generation, MTP depth 1, a five-second timed window, and a cool gate before each timing phase. The official private prompt and serial control were not available locally, so these runs correctly sealed score: null.

prefill · tok/s

decode · tok/s

run	prefill	decode	MTP accepted	checked steps
1	953.12 tok/s	32.50 tok/s	60 / 66	130
2	1,221.77 tok/s	32.42 tok/s	60 / 66	130
3	1,222.74 tok/s	33.60 tok/s	60 / 66	130

Decode varied by 2.01% sample CV. Prefill varied by 13.72% because the first pass was slower; the two subsequent passes were within 0.08%. All three passed the public correctness gate.

the 108.5% claim

composite = prefill gain^0.25 × decode gain^0.75

The leaderboard reports (composite − 1) × 100%. “108.5% faster” therefore means a 2.085× weighted composite—not that both raw throughput numbers individually doubled.

official record progression · composite increase

claim	what the evidence says	verdict
“108.5% faster”	Correct for the current official single-stream composite versus its paired serial control.	supported
“more than twice as fast”	Correct only when “speed” means the weighted composite. Decode carries 75% of the score.	needs context
same output	The target model verifies proposals, but the official token gate permits up to 10% trajectory divergence. It is not a bit-identical-output claim.	needs context
general serving win	Not established: the ranked shape is one stream, one 1K prompt, 128 checked decode steps, and no HTTP serving layer.	not measured
works on a Spark	Yes. My independent build loaded, ran, passed correctness three times, and returned to idle cleanly.	confirmed

The earlier 105.5% post was accurate when published. The board moved again within hours: the same promoted commit now holds 108.5%, with 1,232.2 prefill tok/s and 29.6 decode tok/s on the official private timing shape.

absolute speed in context

cuda.fast was 6.1× faster than my earlier llama.cpp GGUF decode result. The vLLM and SGLang rows used NVFP4 and a 256K deployment sweep, while cuda.fast used its 1K public harness. This chart is orientation, not a controlled engine ranking.

decode · tok/s different benchmark shapes

prefill · tok/s different benchmark shapes

memory trick

The model body is 103.69 GiB before the draft head and working memory. The engine avoids an impossible all-resident allocation by treating the huge per-layer n-gram embedding table as a sparse lookup file.

launch memory plan · GiB

The launch gate required 93.46 GiB against 116.08–116.22 GiB available. The PLE segment is shown separately because it remains on SSD rather than consuming the resident allocation.

thermals & device behavior

Five-second on-device sampling covered setup, all three runs and final cooldown. The plots below show the 101 samples in the benchmark phases. Gaps mark a new resident-engine window.

temperature · °c GPUNVMe max

gpu power · w board draw

available system memory · GiB available

samples	GPU peak	power peak	GPU util peak	NVMe peak	memory floor	loaded limiter samples
707 total · 101 benchmark	56°C	60.35 W	96%	49.85°C	31.91 GiB	0

what I would—and would not—repeat

The engineering is substantive. The SSD split is model-aware, the resident process removes repeated 100 GB loads, and MTP converts a 90.9% accepted draft stream into fewer expensive target-model rounds.
The official comparison is well controlled. The serial denominator is measured beside the candidate, not copied from an old blog post. Calibration is a health check, not the denominator.
The headline needs its noun. “108.5% faster single-stream composite” is precise. “The model is 108.5% faster” invites a broader interpretation than the benchmark supports.
My run validates the engine, not the leaderboard score. The local harness intentionally cannot seal an official score without the organizer's reference tree, calibration and private golden.
It is not yet a drop-in server benchmark. There is no concurrency sweep, 256K context test, HTTP scheduling measurement or long-running production soak in this result.

method & sources

Machine: one NVIDIA DGX Spark, GB10, 128 GB unified memory, CUDA 13.0, driver 580.173.02. Candidate repository fb49cb2; pinned ds4 engine 5f36517; benchd source 206b71b. The model and harness hashes matched across all three runs. Seventy-nine adapter tests also passed.

Sources: official cuda.fast leaderboard and scoring description · public engine, contract and harness · Unsloth GGUF checkpoint · my earlier vLLM, SGLang and llama.cpp comparison.