Qwen-3.8-Next-Flash Ngram Hot-Swappable Knowledge Injector for llama.cpp
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Llama 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
Qwen架构的一项新修改允许Ngram PLE表在一定限制下作为长期知识库运行。这个名为“Ngram知识注入器”的工具可在GitHub上获取，用于创建表补丁。开发者认为，这可能为低成本训练和创建具有即时热插拔长期记忆的模型提供新途径，尽管目前实现起来并不容易。
Looking into the new Qwen architecture, I was curious if you could modify the Ngram PLE Table to make it work like a long-term knowledge database. It turns out that, with some limitations, you can.
I coded a small modification to llama.cpp to modify the table in-memory, allowing you to patch it with new data in real time. The PLE table is updated on every prompt, so now you can hot-swap parts of it without reloading the model.
The limitation is that it’s hard to control the output reliably, as the embeddings are injected early in the layers. However, with some techniques, you can influence the model’s output with simple modifications, as the example shows.
- The Ngram knowledge injector (a kind of compiler to create the table patches) here: https://github.com/ortegaalfredo/ngram-knowledge-injector
There are some limitations in the project, as the PLE table needs to be memory-mapped into memory (this is the default in llama.cpp), and I have only tested it with q8 quantization, so you need quite a bit of memory to test this.
Can this be used as a new way of low-cost training? Perhaps. Its not easy at the current state but with simple modifications, I think you could easily create models with long-term instantaneously hot-swappable memory.