A Mathematical Framework for Transformer Circuits (2021)

Transformer language models, such as GPT-3 and LaMDA, are increasingly used in real-world applications. However, their scaling and open-ended nature can lead to unexpected and harmful behaviors, with creators and users discovering new capabilities, including problematic ones, years after training. This framework aims to provide clarity and simplicity, making superficial changes and largely ignoring biases and layer normalization, as these can often be folded into weights or adjacent parameters.

This paper offers a mathematical framework for transformer circuits, unlike previous approaches that often overlook biases and layer normalization by folding them into weights.