How are you guys thinking about context now, and building around it?

A user on reddit.com inquired about how others are approaching context in AI models, particularly given that newer models can process large amounts of information for reasoning. They are interested in understanding the strategies of those with more limited local hosting setups. The user themselves runs a small setup, utilizing 32GB VRAM on their main PC and an older machine with 8GB running a 9B Qwen model for compaction and long-term memory sorting.

This discussion uniquely focuses on context management strategies for AI models on local, resource-constrained setups, unlike broader discussions on large-scale deployments.