How many agents can 2×4090 actually run at once? Three weeks of llama.cpp concurrency data — soft cap 5 @ 64k, hard cap 9, and why.

A CTO benchmarked llama.cpp concurrency on a local Qwen stack using 2x4090 GPUs over three weeks. The study aimed to determine the maximum number of agents that could run simultaneously and their optimal context. Results showed a soft cap of 5 agents and a hard cap of 9 at 64k context. q8_0 quantization proved most efficient, offering similar quality and speed to f16 while being 1.4 GiB cheaper and consuming 29.9 GiB VRAM at 65k context.