experimental / august 2026
Qwen3.8 on DGX Spark: a 256K context sweep
My homelab includes an NVIDIA DGX Spark — a GB10 desktop box with 119 GB of unified memory shared between the Grace CPU and Blackwell GPU. When Qwen shipped Qwen3.8-27B-FP8 in August 2026, I benchmarked it under vLLM at every context size from 512 to its full 262,144-token window: one request per size, 256 new tokens, greedy decoding.
The run was instrumented end to end — GPU and CPU-zone temperatures, per-core clocks,
and board power sampled every 5 seconds by a headless logger built on my
dgxmon monitor's collectors. Every chart below is interactive; hover for
exact readings.
the punchline
Single-request decode on this box is a memory-bandwidth problem. The GB10's unified LPDDR5x moves ~273 GB/s, and decoding streams every active weight byte once per token — so the ceiling is simply bandwidth ÷ bytes-per-token. Measured decode rates from this sweep and my earlier Qwen3.6 runs on the same box land right on that roofline, which is exactly why the local-inference meta has converged on MoE:
| model | bytes read / token | roofline | measured @ 1K ctx |
|---|---|---|---|
| Qwen3.6-27B dense · bf16 | ~54 GB | ~5 tok/s | 4.3 tok/s |
| Qwen3.8-27B dense · FP8 | ~28 GB | ~9.8 tok/s | 8.4 tok/s |
| Qwen3.6-27B dense · 5.5-bit | ~19 GB | ~14 tok/s | 10.3 tok/s |
| Qwen3.6-35B-A3B MoE · bf16 | ~7 GB active | ~35–40 tok/s | 29.2 tok/s |
| Qwen3.6-35B-A3B MoE · int8 | ~3.5 GB active | ~70 tok/s | 36.9 tok/s |
Dense throughput scales almost linearly with 1/bytes; a bigger MoE outruns a smaller dense model 5–7× because it only touches ~3B active parameters per token. Prefill is the counterpoint — it's compute-bound, and the dense 27B still ingests 400–540 tok/s at long context.
thermals & clocks
44 minutes on a shared timeline. Shaded columns mark benchmark phases; each begins with an engine boot (model load + CUDA graph capture) before its prompts run. Amber strips on the clock panel mark samples where the GPU reported a ⚠ sw power cap throttle reason.
temperature · °c GPU CPU zone (max) NVMe
clock frequency · mhz CPU mean per-core range GPU SM
gpu power · w board draw
throughput vs context
Prefill rate climbs with prompt length and holds roughly flat past 64K. The end-to-end generation rate (256 tokens ÷ total wall time, prefill included) falls as prefill dominates.
prefill · tok/s
end-to-end generation · tok/s
results
| context target | prompt tokens | wall time | prefill tok/s | gen tok/s (e2e) |
|---|
findings
- The full 262,144-token window fits and completes in 119 GB of unified memory at gpu-memory-utilization 0.90 — Qwen3.8's hybrid linear-attention design (3 of every 4 layers linear) keeps KV-cache growth modest.
- The GB10 is power-limited, not thermally limited. GPU temperature plateaus at 84–85 °C during sustained prefill with the board holding ~90 W and intermittent ⚠ sw power cap events.
-
CPU cores never throttled. The hottest ACPI CPU zone touched 90.7 °C during
the 256K prefill, but no core raised
perf_limitedand per-core frequencies held their normal 2.8–4.0 GHz envelope throughout. - Prefill peaks near 540 tok/s at 128K and eases to ~460 tok/s at 256K; end-to-end generation falls from ~8 tok/s at 1K context to 0.45 tok/s at 256K as prefill dominates the 9.5-minute wall time.