Infermeld: a Linux kit for running one GGUF across AMD + NVIDIA GPUs with llama.cpp

Infermeld is an open-source Linux kit designed to run a single GGUF model across both AMD and NVIDIA GPUs using llama.cpp. It was tested with an AMD RX 6900 XT (16GB) and an NVIDIA RTX 3080 (10GB) using the Qwen3.6-35B-A3B, UD-Q4_K_M GGUF model. The kit utilizes Vulkan and CUDA backends in split mode, reserving 8,192 tokens for context, and supports loading and short completions with MTP both off and on.

This kit is among the first to enable running a single GGUF model across both AMD and NVIDIA GPUs simultaneously, unlike previous solutions that typically target one vendor.