Neural networks are clean

Modern language models are more interpretable than commonly thought. Research like Anthropic’s “Towards Monosemanticity” (2023) and OpenAI’s “Language Models Can Explain Neurons in Language Models” (2023) show that while some neurons are hard to interpret, a significant portion can be understood. The concept of recursive generalization, where similar things are grouped together, suggests that interpretability is possible by examining how the network learns and organizes information, offering a fractal understanding of its functions.