DeepSeek V4 Flash 0731 at 27+ t/s decode on Strix Halo — Vulkan + DSpark full guide

Been benchmarking DSv4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128GB LPDDR5X) for the past week. Figured I'd share what actually works and what doesn't — there are a lot of gotchas on this hardware.

Results

Best client-side observation (bench-kv.sh streaming, 4096-token generation). Server-side timing (predicted_per_second) for similar runs consistently shows 23-24 t/s. See gotcha #7.

Metric Value Decode (best client-side, 4096 tok) 26.76 t/s Decode (server-side typical) 23-24 t/s Decode (peak 3s window) 35.27 t/s Prefill 236 t/s (2209-token prompt) Plain decode (no spec, same engine) 18.33 t/s (llama-bench) Plain prefill (no spec, same engine) 254 t/s (llama-bench) DSpark acceptance 0.586, mean accepted len 3.93 Context 131,072 (q8_0 KV) GPU util ~92%, CPU ~1%

Cross-platform comparison

Both Strix Halo and DGX Spark are unified-memory APUs with similar LPDDR5X bandwidth (~256 GB/s vs 273 GB/s).

Platform Engine Backend Spec Decode Prefill @2K Strix Halo llama.cpp v0.6.1 Vulkan none 18.33 t/s 254 t/s Strix Halo ds4 (upstream) ROCm none 12.5 t/s 122 t/s DGX Spark ds4 (upstream) CUDA none 14.2 t/s 392 t/s DGX Spark Entrpi/ds4 fork CUDA none 20.0 t/s ~960 t/s Strix Halo llama.cpp v0.6.1 Vulkan DSpark 26.76 t/s ~236 t/s DGX Spark Entrpi/ds4 fork v0.5.6 CUDA DSpark 27.3 t/s ~960 t/s Note: ds4 (antirez's engine) is ROCm/HIP-only on AMD — no Vulkan backend. The 122 t/s ROCm number is artificially low because ROCm support on gfx1151 is still early (see gotcha #3). Prefill is unaffected by speculation (DSpark is decode-only). Decode is bandwidth-bound — context depth barely affects it.

Decode is a dead heat. All plain decode numbers cluster in the 12-20 t/s range regardless of engine or platform — bandwidth-bound. With DSpark, both platforms reach ~27 t/s. The same-engine DSpark boost is similar: 1.46× on Strix Halo (26.76 / 18.33, llama.cpp Vulkan) vs 1.37× on DGX Spark (27.3 / 20.0, fork CUDA). Speculation is the only lever on bandwidth-bound decode, and it behaves the same on both platforms.

Prefill gap can't be cleanly decomposed into hardware vs software. No single engine runs optimally on both platforms (ds4 is ROCm-only on Strix where it gets 122 t/s; llama.cpp is Vulkan-only). The best available comparison: llama.cpp Vulkan 254 t/s on Strix vs fork D2R CUDA 960 t/s on DGX = 3.8× gap. The fork's D2R optimization is a clean 2.45× over its own ds4-CUDA baseline (960 / 392, same hardware, same backend). If Vulkan's MoE shaders improved by a similar factor, Strix would project to ~568 t/s (232 × 2.45) — at which point the residual gap would be ~1.7×. But that's a projection, not a measurement. The honest takeaway: MoE prefill doesn't scale with raw FP16 TFLOPS (the compute ratio is 4.2× but the real gap is 3.8×), and most of the gap is closeable with better shaders.

Caveats: different model quants (IQ3_XXS ~98GB on Strix vs Q2 ~81GB on DGX Spark), different engines, different draft precision (bf16 vs Q2K). Sources: Strix Halo numbers are mine. DGX Spark DSpark numbers from Entrpi/ds4-on-spark (v0.5.6, 2026-08-08).

Side note: the Q2K drafter that crashed on llama.cpp (invalid token = -1) works fine on the ds4 engine with 74.6% acceptance. That's a llama.cpp-specific bug, not a model quality issue.

Config

- Target: Unsloth UD-IQ3_XXS (~98GB, 4-file split)

- Draft: DSpark bf16 drafter (~11GB)

- KV: q8_0 (not f16 — see below)

- Backend: Vulkan via Nathan's v0.6.1 build (bundled RADV, no ROCm)

- OS: CachyOS, kernel 7.1.6, Mesa 26.1.6

bash llama-server \ -m DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf \ -md DSV4-Flash-DSpark-draft-bf16.gguf \ -ngl all -ngld all \ -fa on \ -ctk q8_0 -ctv q8_0 \ -c 131072 -np 1 \ -b 2048 -ub 2048 \ --spec-type draft-dspark \ --spec-draft-n-max 64 \ --jinja --host 127.0.0.1 --port 8080

Boot params: amd_iommu=off amdgpu.gttsize=126976 ttm.pages_limit=32505856 ttm.page_pool_size=32505856

Power: z13ctl+ profile (CPU boost off, min freq pinned) — keeps the tablet cool and quiet for long runs, and decode is bandwidth-bound so the downclock is free.

VRAM: 4GB BIOS carve-out. GTT covers the rest. Total committed ~109GB (98+11), leaving ~19GB for KV + compute.

Gotchas (learned the hard way)

1. q8_0 KV beats f16 on long gens — but loses on short ones. The crossover surprised me. Measured same engine, same model, same prompt shape:

KV type 1024 tok 4096 tok f16 65k 21.99 t/s 19.32 t/s q8_0 131k 20.70 t/s 22.74 t/s At short context, q8_0 is ~6% slower — the per-element dequant overhead dominates when the KV cache is small. At long context, KV bandwidth becomes significant and q8_0's halved bytes-per-step outweighs the dequant cost (+18%). Also doubles context (131k vs 65k). The 4096-tok q8_0 run also had higher draft acceptance (0.575 vs 0.496) — that may be run-to-run variance or a real KV-quality effect; can't isolate from a single pair of runs.