An interesting paper on Continuous (Live) Learning: "Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data"

A new paper, "Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data," explores continuous learning for LLMs. Building on previous TTT research like Titans and recent Dynamic LoRA adapters, this work introduces a similar concept (SHINE). However, it diverges by generating an adapter that is directly applied to existing weights on-the-fly, rather than using LoRA. The authors employ a rigorous Bayesian belief for uncertainty, though the method could potentially be extended to a continuous version by using standard ML techniques.