2×RTX 3090 + EPYC box running qwen3.8-flash-next at ~38 tok/s
A user is running Qwen3-Flash-Next (177B / ~6B active MoE, IQ4_XS) on Ilama.cpp with a system equipped with two RTX 3090 graphics cards and an EPYC processor. This configuration achieves approximately 38 tok/s in single-stream mode. However, when processing two parallel requests simultaneously, performance significantly drops to about 4 tok/s per request. The user aims to improve performance when multiple agents run in parallel.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Sep 12, 2026, 16:52 UTC
IngestedOffset at this time: UTC+0Sep 13, 2026, 15:01 UTC
- Published
- Sep 12, 2026, 16:52
- Ingested
- Sep 13, 2026, 15:01
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
What I have:
- CPU: EPYC 7551 (32c/64T, Zen 1)
- Board: Supermicro H11SSL-i (SP3), Rev 2.0
- RAM: 128 GB DDR4-2133 (all 8 channels full)
- GPU: 2x RTX 3090 (48 GB total, PCIe 3.0)
- 1500 W PSU
What I run:
- Qwen3-Flash-Next (177B total / ~6B active MoE, IQ4_XS) on Ilama.cpp. Experts live in system RAM, hot ones cached in VRAM. Single stream = 38 tok/s. Two parallel requests drop to ~4 tok/s each.
Budget:
~$800. Realistically that's either one more RTX 3090 or a CPU upgrade (a Zen 2 "Rome" EPYC drops into the same board). A new motherboard is out of budget i think for now.
Which gives more inference speed for this setup - adding the 3rd 3090, or swapping to a faster/newer CPU?
And would more/faster RAM matter here? Curious what people running similar rigs have actually measured.
I am also interested in having multiple agents running at the same time, which currently slows it down heavily, so keeping the performance at multiple agents parallel would be a huge boost as well!