Back
RCreddit.com

Expert expansion with llama.cpp

View original
LlamaModel release
llama
Time & source
Published
09/06, 18:27
Ingested
09/06, 23:00
Source type
Dev community
Tier
Community
Source status
Healthy
Tier is a per-source editorial setting, not a per-item score.
AI summary

A developer has created a custom branch of llama.cpp to support Expert expansion with MOE models, utilizing Glm 5.3 flash. The developer reports that this new version, named "moex-expansion," performs better than their previous DS4 version on metal. They are now seeking feedback from other platforms and with different models to further evaluate its performance and stability.

With the help of Glm 5.3 flash I built a custom branch of llama.cpp in order to support Expert expansion with MOE models, I've tested only on metal and It works better than my DS4 version, i need feedback from other platforms, and different models.