What pi.dev plugin do you suggest for context, compaction and memory management of local models?
A user is seeking suggestions for a pi.dev plugin to manage context, compaction, and memory for local models. They are running a Qwen3.8:27b setup on an RTX 5080 16GB, utilizing llama.cpp with a nvfp4 version of the model. Their setup involves llama-server.exe with specific parameters like --jinja, --chat-template-file, --reasoning-format deepseek, and -c 49152 for context size.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Sep 12, 2026, 21:19 UTC
IngestedOffset at this time: UTC+0Sep 13, 2026, 15:01 UTC
- Published
- Sep 12, 2026, 21:19
- Ingested
- Sep 13, 2026, 15:01
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
I have been battling with my Qwen3.8:27b setup on my rtx 5080 16gb. I am using llama.cpp to run a nvfp4 version of qwen3.8:27b
llama-b10699-bin-win-cuda-13.3-x64\llama-server.exe -hf williamliao/Qwen3.8-27B-NVFP4-GGUF:NVFP4 --jinja --chat-template-file d:\qwen\chat_template.jinja --reasoning-format deepseek --host 127.0.0.1 --offline -np 1 --threads 16 --threads-batch 16 --log-verbosity 4 --no-log-prefix --no-log-timestamps --no-mmproj-offload --mmproj D:\OllamaModels\blobs\sha256-ac3714bfdddeca31351f2752bf1a63f266f4df87c0b68c895e44945ca704448e --load-mode none --cache-type-k q4_0 --cache-type-v iq4_nl --flash-attn on -b 512 -ub 512 -c 49152 -a qwen3.8:27b
So that is giving me a 48k context, and a speed of around 12t/s
My problem is that pi.dev fails to compact at the right times, and then the model stops early.
I have tried to use
pi install npm:max-context //this doesnt work properly at all
pi install npm:pi-observational-memory //this one works kind of, I had it set to compact on threshold. It managed to auto compact 4 times, with a 0.75 ratio set, but it still missed the mark and failed.
pi install npm:pi-blackhole //seemed promising, but failed to compact at all, I am not sure what I set wrong.
if I look at the https://pi.dev/packages page, there are lots of plugins for either context management, compaction, memory management, or both. there are way too many for me to just randomly keep trying. other AI has been very unhelpful. Google, OpenAI, Copilot, all full of nonsense. A lot of these plugins seem to be aimed at frontier model management.
I am also disappointed that compaction settings are best suited to the specific model being used and its context window, but the built in tools in pi.dev architecturally keep the compaction settings completely separate from model settings, which seems a bit of an issue if you change models often.
I may want to use qwen3.8 like above, but not necessarily for everything. I also want to use gemma, other qwen models, etc depending on my use case.
I dont want to have to manually resetup context settings in pi.dev for every different model.
What do you guys suggest, especially for models that have smaller context windows due to my vram shortage?