Now the bottleneck is compression / memory (and a little on input filtering)

The current bottleneck in AI development is compression and memory, rather than short-term memory, where computers excel. While large context windows like 258k tokens are a temporary fix, true progress lies in efficient long-term memory. Humans can recall specific details from a 10 million token codebase over 10 years, a feat AI struggles with. Solving this could enable AI to surpass human intelligence, allowing skills to be stored and automatically applied, potentially automating a significant portion of white-collar jobs by enabling cheap, fast, and accurate recall of information.

This post uniquely argues that AI's bottleneck is memory compression, unlike common discussions focusing on short-term memory or context window size.