2x R9700, 64 GB DDR5 is an absolute beast machine with vLLM Radiance / R9V and Qwen 3.8 27b and Flash next

I've been tinkering with local LLMs since the beginning of the year when I had an Intel Arc B580 and 32 GB of DDR5. Curiosity got the best of me and I bought the first R9700 about half a year ago, also because I wanted to upgrade my gaming graphics for 4k. As the 5090 was about 3 times as expensive, I had a "sweet spot", kind of. On the last prime days, I found a X870E mainboard for ~150 € below the standard price, and it got to my head that I can use an upgraded machine for gaming and local inference tinkering.

Anyways. Fast forward to this week, I now have the following setup

- Ryzen 7500F

- 64 GB DDR5 CL40 6400 MT/s

- Asus ProArt Creator X870E

- 2x R9700 32 GB, each running at PCIe 5.0 x8 (Gigagbyte)

- Currently running ubuntu on an old Samsung EVO 860 1 TB drive; this will become intersting for the ngram / PLE offload; I have Windows and the gaming related stuff on a gen4 NVMe, but will soon add another Gen 5 NVMe with decent random reads

The only issue that I can report so far is that one of the cards runs quite hot, so I will definitely implement power limiting to 210 W and some light undervolting. The other card runs 10-15 °C cooler.. Case is a purebase 501 with 4 fans, 2 intake in front, one back and top for output.

Now long story short I wanted to give some results of Qwen 3.8 27b FP8 and MXFP4, as well as Qwen 3.8 flash next after the first day tinkering with it. What I found super interesting is that the SATA SSD does not seem to be super terrible when using Qwen 3.8 flash next.

Considering the whole build costs ~4k €, or more than 1k less than a single RTX 5090 with 32 GB, I kinda like this setup price/performance wise. Next step is checking context degradation / KV quants. I am using local inference mostly for deep research, summarization, image creation, light coding and non-trivial data analysis

Cheers

Qwen3.8 benchmarks on 2× Radeon AI PRO R9700

Hardware: 2× AMD Radeon AI PRO R9700 32 GB, 61 GiB system RAM Benchmark: BetterBench 0.2.2, corpus v1.0, single-stream, greedy decoding, 2 warm-ups + 10 measured runs per category, 8k benchmark context.

Model   Weight format   Runtime   Server context   Max sequences   Speculative decoding   Weighted decode median   ITL 1% low   TTFT p50   Prefill ~2k   Prefill ~4k   Prefill ~7k          Qwen3.8-27B   Quark AWQ MXFP4   vLLM Radiance, TP2   131,072   1   MTP, up to 8 tokens   111.4 tok/s   77.9 tok/s   81 ms   4,224 tok/s   4,322 tok/s   4,410 tok/s       Qwen3.8-27B   Native block FP8   vLLM Radiance, TP2   16,384   8   MTP, up to 8 tokens   87.6 tok/s   61.9 tok/s   73 ms   4,134 tok/s   4,329 tok/s   4,305 tok/s       Qwen3.8-Flash-Next   UD-IQ4_XS GGUF   R9V/vLLM, TP2, tiered expert offload   131,072   1   MTP, 2 tokens, FP8 draft   35.4 tok/s   27.3 tok/s   290 ms   1,727 tok/s   1,986 tok/s   1,925 tok/s
- Qwen 3.8 27b in FP8 and AWQ MXFP4 served with vLLM Radiance

- Qwen 3.8 Flash next served with vLLM / R9V fork

- Decode metrics come from the 10-pass standard run.

- Prefill measurements use cold, nonce-prefixed prompts.

- Prompt-token medians for the prefill columns were 1,556, 3,024 and 5,226 tokens.

- No concurrency sweep was included in these results.

- I expect decode of Flash next to increase a bit when an NVMe is used, and, as I am writing this and checked, I found EXPO was not enabled........oh my god I swear I turned it on when I updated the bios yesterday