Qwen-3.8-Next-Flash Ngram Hot-Swappable Knowledge Injector for llama.cpp
Heat trend
The percentage is based on available heat signal, not comment count or independent people.
Llama model activity is surfacing — worth tracking for capability changes, ecosystem impact, and availability.
A new Qwen architecture modification allows the Ngram PLE Table to function as a long-term knowledge database, despite some limitations. This "Ngram knowledge injector" tool, available on GitHub, creates table patches. The developer suggests this could enable low-cost training and the creation of models with instantaneously hot-swappable long-term memory, though it's not easy in its current state.
Looking into the new Qwen architecture, I was curious if you could modify the Ngram PLE Table to make it work like a long-term knowledge database. It turns out that, with some limitations, you can.
I coded a small modification to llama.cpp to modify the table in-memory, allowing you to patch it with new data in real time. The PLE table is updated on every prompt, so now you can hot-swap parts of it without reloading the model.
The limitation is that it’s hard to control the output reliably, as the embeddings are injected early in the layers. However, with some techniques, you can influence the model’s output with simple modifications, as the example shows.
- The Ngram knowledge injector (a kind of compiler to create the table patches) here: https://github.com/ortegaalfredo/ngram-knowledge-injector
There are some limitations in the project, as the PLE table needs to be memory-mapped into memory (this is the default in llama.cpp), and I have only tested it with q8 quantization, so you need quite a bit of memory to test this.
Can this be used as a new way of low-cost training? Perhaps. Its not easy at the current state but with simple modifications, I think you could easily create models with long-term instantaneously hot-swappable memory.