Back
RCreddit.com

gfx906-llama-cpp: New PP/TG gains for MI50/MI60/Radeon VII/AMD GCN

View original
Llama
llamagfx906mi50
Time & source
Published
09/05, 14:42
Ingested
09/05, 20:00
Source type
Dev community
Tier
Community
Source status
Healthy
Tier is a per-source editorial setting, not a per-item score.

Time for another update! We have been busy and managed to improve the gains substantially (mostly from exploring existing llama cpp PRs and adopting relevant things).

Among other things the README.md was also appended to provide a better overall picture of what’s in the fork, why and from whom.

metric upstream t/s fork t/s gain prefill PP16384 332.5 ~410 +23% 120k deep fill 231.4 ~264 +14% TG 13.6 ~15.1 +11% (parity pre-mirror) context cannot fit 250k on 40 GB tight-fit machinery outputs - - bit-identical (sha + token-for-token) https://github.com/milpster/gfx906-llama-cpp/blob/master/README.md

(Yes i made this with AI)