Is it possible to run it with a combined memory setup: 16 GB VRAM + 64 GB RAM + SSD for offloading n-grams?
Llama 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
一位用户询问了关于在组合内存设置下运行语言模型的可行性，该设置包括16 GB VRAM、64 GB RAM以及用于卸载n-gram的SSD。他们详细说明了硬件配置，例如RTX 5080和Ryzen 7 9800 X3D，并表示偏好在Windows 11上使用llama.cpp。尽管使用了特定的llama-server命令，并设置了-ngl 99和-b 512等参数，用户报告的性能仅为每秒6个token，认为其无法使用。
Hardware: rtx 5080 16 gb vram; 64 gb ram ddr5 6000hz; ssd with unlimited memory; ryzen 7 9800 x3d. OS: Windows 11 Software: I’d prefer llama.cpp, but it’s not a strict requirement; I’ll use whatever you suggest, as long as it works on Windows.
My attempts to run it with llama.cpp:
llama-server ^ -m "F:.lmstudio\models\unsloth\Qwen-Next\Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf" ^ -c 10000 ^ --n-gpu-layers 999 ^ -b 512 ^ -ub 512 ^ --fit off ^ --parallel 1 ^ --jinja ^ --flash-attn auto ^ --load-mode mmap ^ --no-host ^ --override-tensor "per_layer_token_embd.weight=CPU"
and 2nd attemtp:
llama-server ^ -m "F:.lmstudio\models\AtomicChat\Qwen3.8-Flash-Next-GGUF\Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64-00001-of-00033.gguf" ^ -c 10000 ^ -ngl 99 ^ -b 512 ^ -ub 512 ^ --fit off ^ --parallel 1 ^ --jinja ^ -fa on
and i got 6 t/sec, its just unusable