Explaining Attention with Program Synthesis

The same day I discovered Tracr, this paper dropped. Very interesting and potentially accelerates LLM training significantly. The idea of programmable attention seems promising.