experimental / september 2026
Qwen3.8-Flash-Next on DGX Spark: three runtimes, one box
Qwen3.8-Flash-Next is a 125B-parameter MoE with 6B active parameters, a 51B n-gram embedding table, and a 4B MTP module. I ran it on one NVIDIA DGX Spark under vLLM and SGLang using the same NVFP4 checkpoint, then added Unsloth's UD-Q4_K_XL GGUF under llama.cpp.
Every accepted request used a fresh prompt: prefix caching was disabled across all three stacks. The sweep covers ten context targets through 262K, four concurrency levels, five correctness canaries per engine, and five-second thermal, clock, power, memory, swap, and page-fault telemetry.
the punchline
The two CUDA-native servers are close on fresh-prompt prefill. SGLang leads single-stream decode, while vLLM scales best at the top of this concurrency ladder. The GGUF build fits with far more headroom, but llama.cpp without MTP is much slower at long context.
| engine | 256K TTFT | 256K prefill | 256K decode | c8 aggregate | memory floor |
|---|---|---|---|---|---|
| vLLM · NVFP4 | 110.68 s | 2,132 tok/s | 35.36 tok/s | 162.7 tok/s | 5.86 GiB |
| SGLang · NVFP4 | 108.97 s | 2,165 tok/s | 42.04 tok/s | 151.5 tok/s | 6.83 GiB |
| llama.cpp · UD-Q4_K_XL | 778.30 s | 303 tok/s | 5.37 tok/s | 54.1 tok/s | 24.99 GiB |
The llama.cpp row is a deployment comparison, not an engine-only comparison: its quantization, format, kernels, and speculative-decoding configuration differ.
throughput vs context
One fresh-prompt request per target, 256 generated tokens, greedy decoding, EOS ignored. The x-axis uses log₂ spacing. Hover a point for exact prompt count, time, and throughput.
prefill · tok/s
decode · tok/s
time to first token · seconds log scale
end-to-end generation · tok/s
context results
| target | actual | engine | TTFT | prefill | decode | E2E |
|---|
throughput vs concurrency
Each point is the median of three fresh-prompt runs. Every stream used 918 prompt tokens and returned exactly 256 output tokens.
aggregate output · tok/s
concurrency medians
| streams | engine | aggregate | per stream | TTFT p50 | TTFT p95 |
|---|
thermals & clocks
Select an engine to inspect its accepted timing phases. Temperature, clocks, and board power share a synchronized crosshair. Shaded regions distinguish context from concurrency measurements; gaps mark cooldown or a server restart.
temperature · °c GPUCPU zone maxNVMe max
clock frequency · mhz CPU meanGPU SM
gpu power · w board draw
timed-measurement summary
| engine | GPU peak | CPU peak | NVMe peak | power mean | power peak | GPU util mean | timed throttle |
|---|---|---|---|---|---|---|---|
| vLLM | 77.0°C | 82.0°C | 59.9°C | 54.8 W | 70.9 W | 93.7% | 0 |
| SGLang | 77.0°C | 82.7°C | 61.9°C | 54.6 W | 67.7 W | 95.6% | 0 |
| llama.cpp | 81.0°C | 90.0°C | 64.9°C | 62.1 W | 76.9 W | 92.5% | 0 |
test shape
| vLLM | SGLang | llama.cpp | |
|---|---|---|---|
| version | 0.1.dev20073 · 8e685d198 | 4ccff141d | 0.4.0-dev · 9dcf84e5a |
| checkpoint | RadixArk NVFP4 | same NVFP4 | Unsloth UD-Q4_K_XL |
| context setting | 262,144 | 262,144 | 262,144 |
| speculation | MTP · 2 tokens | NEXTN/EAGLE · 3 steps | disabled |
| prompt cache | disabled | radix disabled | disabled per request |
| PLE / weights | file-backed mmap | file-backed · 8 GiB RSS cap | mmap GGUF |
The NVFP4 checkpoint occupied 126 GiB and all 208 LFS objects passed SHA-256 verification. The four-shard GGUF occupied 104 GiB and every shard hash was recorded. llama.cpp MTP stayed off because its open parallel-request correctness bug can contaminate outputs between slots. The observed cold boot times are not ranked: page-cache state differed materially after the 104 GiB GGUF download.
findings
- SGLang wins decode. It remained roughly 18–22% faster than vLLM across the context sweep, while fresh-prompt prefill was within a few percent at the four largest sizes.
- vLLM scales farther at concurrency eight. SGLang led aggregate throughput at one, two, and four streams; vLLM won the eight-stream row 162.7 to 151.5 tok/s.
- The GGUF buys headroom, not speed. It completed 235,926 prompt tokens with 24.99 GiB still available, but 256K TTFT was 7.0× vLLM's and decode was 84.8% slower.
- Near-full context works in all three stacks. Each completed 235,926 prompt tokens plus 256 generated tokens. I did not send an exactly 262,144-token prompt, so this does not prove the absolute boundary.
caveats & sources
One Spark, one observation at each context size, and pinned development images. This is not a BF16, FP8, or model-quality comparison. Client-observed TTFT includes HTTP, scheduling, and first-token work. The GGUF row changes more than the serving engine. Greedy raw completions were chosen for repeatability.
Sources: Qwen model card · Unsloth GGUF card · SGLang single-Spark recipe · vLLM recipe · llama.cpp parallel MTP issue · llama.cpp exact-262K issue.