Two 96 GB Ascend cards crun Qwen3.8-flash-next hardware notes, vLLM work, benchmarks, and what is next

A developer built a local inference machine using two Huawei Atlas 300I Duo cards, each with 96 GB of device memory. These cards are not direct CUDA replacements but offer a cost-effective, passive, dual-accelerator PCIe solution. Initially, the setup produced slow, incoherent output, but after optimization, it achieved 48.93 aggregate tok/s for long reasoning generations with Qwen3.8-flash-next. While not outperforming NVIDIA in per-request speed, this demonstrates the potential of the Ascend platform for large model inference.