$2800 rig with 8x Radeon Pro V620 (256 GB VRAM) + custom vLLM fork = Qwen3.8-Flash-Next at 60 to 100 t/s decode and 3000+ t/s prefill
A user built a $2800 rig equipped with 8 Radeon Pro V620 graphics cards, each with 32 GB VRAM, totaling 256 GB VRAM. Combined with a custom vLLM fork, this setup demonstrated exceptional performance when running Qwen3.8-Flash-Next, achieving decode speeds of 60 to 100 tokens per second and prefill speeds exceeding 3000 tokens per second. This configuration utilized vLLM's PP=4 setting, W4A16 quantized routing experts, and MTP functionality with 3 token drafts.
时间与来源
时间显示为 UTC
显示时区：UTC
本地时区尚不可用，暂时显示 UTC。
发布当时偏移：UTC+02026年10月8日 17:12 UTC
收录当时偏移：UTC+02026年10月9日 02:00 UTC
- 发布
- 2026年10月8日 17:12
- 收录
- 2026年10月9日 02:00
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
档位是按信源手工设定的编辑判断，不是逐条打分。
讨论趋势
百分比基于采集到的讨论信号，不代表新增评论数或独立参与人数。曲线仅用于同一话题在不同时段的比较。
Post title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterprise cloud gaming cards with 32 GB VRAM.
(Ignore the RTX 4090 on the side, it's just used for stuff like image/video gen models, no LLMs)
But I bought these cards a couple months ago as a gamble to see if I could build a big VRAM rig with usable speed for relative peanuts.
I was struggling with llama.cpp for a long time, but the prefill was pretty bad (around 350-450 t/s average with this same model) and vLLM just didn't work on the cards. Plus llama.cpp just sucks at concurrency.
I'd been planning to sell the cards lately because this wasn't going to work for my use case, but then decided to see if I (Claude) could make a vLLM fork that both works with the cards and actually gets good speeds out of them. I had it build/test/iterate on custom RDNA2 kernels.
Problem solved! It worked out way better than I expected. I thought maybe I'd hit 1000 t/s prefill with QFN at best, but this is something like 800% faster than llama.cpp was managing.
I'm going to have it continue optimizing and see how it goes, and make sure DeepSeek and GLM-5.3-Flash work as well.
llama-benchy results below with concurrency = 1 and vLLM running with PP=4 (no tensor parallel here) with orcarouter's uncensored QFN which I quantized. Routed experts are W4A16 and everything else remains at BF16. MTP enabled with 3 token drafting.
https://preview.redd.it/4223w82yz9uh1.png?width=666&format=png&auto=webp&s=a15e5d1f5664fcb894a746a54a501eb8f6637a57